Campfire port in Java is faster then Rust. Some thoughts about JEP 544 (AOT Code Compilation)

[u/ThaJedi]

DHH is porting campfire to Rust. I saw port in C# so I was wondering about Java performance. I asked Claude to do the migration, and even though the first version was faster, it was very far from idiomatic Java. I tried to close the gap, but some compromise had to be reached in order to make this work as a drop-in replacement.

Also, some typical Java approaches were just slower. Probably both the Rust and Java implementations could probably do better in terms of performance and being closer to how idiomatic code is written by everyday developers

Stack is Java + quarkus.

Beside just Java migration I tested some new feature like AOT with training introduced in Java 28 (EA build). My main conclusion is - its hard to beat a language that will adjust code to how your app is used. Its more about the JVM than Java.

I tried different combinations of AOT also with Graalvm CE. I didn't put much effort into making graal more optimal. Probably fair comparision would be to also do training run and add EE edition.

What I notice, and it's obvious in hindsight, AOT helps when end usage is same as training usage. When id differs code it optimized for different path and it hurts execution. It's not so visible in benchmark numbers but on smaller and shorter runs it was more visible. JIT needs time to adjust.

Edit: Repo: https://github.com/kgonia/once-campfire-java

[u/TomKavees]

Er.. as much as I want Java to succeed in here, are you sure both versions are doing the same thing? Consistently being >=1.5x faster than native code smells fishy

[u/Augunrik → u/TomKavees]

Even more so against compiled software. Java can be faster but I doubt it is so consistently faster.

[u/ThaJedi → u/TomKavees]

This is expected. JIT should give you better peak throughput than AOT and JVM runtime overhead isn't that big.

[u/PiratesWhoSayGGER → u/TomKavees]

rust is not a faster language - it gives tou tools that can be faster if you utilise them well

I coded in C a lot and it's tedious to write C code that is acutally faster than Java, very tedious

it's vibe coded rust, so no wonder vibe coded java is faster by default

[u/BenchEmbarrassed7316 → u/PiratesWhoSayGGER]

Correction: this is incredibly bad vibe code in Rust. Under normal usage, agents will generate much better code.

I just opened a random file:

https://github.com/basecamp/once-campfire-rust/blob/main/crates/cable/src/naming.rs

fn underscore(word: &str) -> String {
    let chars: Vec<char> = word.replace("::", "/").chars().collect();
    let mut out = String::with_capacity(chars.len() + 4);
    for (i, &c) in chars.iter().enumerate() { /* ...  */ }
    // ...
}

Let me explain: the function takes a reference to a buffer containing a string. .replace allocates memory for a new string and copies the buffer into it, replacing specific characters. Then, .chars() creates an iterator from this string (creating iterators in Rust is inexpensive, and the compiler can sometimes optimize them even better than standard loops). .collect() creates yet another buffer and copies the iterator's output into it - which is completely pointless. However, this is followed by an "incredible" optimization: String::with_capacity(chars.len() + 4). Instead of allocating a buffer of an unknown size, the code explicitly specifies that the buffer should have a size equal to "the length of the string in chars + 4." In other words, to allocate the required amount of memory, it performs an allocation, writes data into it, and then checks the size of the allocated memory. This is absolute nonsense.

I don’t know how he got the AI ​​to generate such bad code.

added: Furthermore, this optimization is incorrect because the input data is in UTF-8, yet the buffer allocation is based on the number of characters rather than the number of bytes. Once again, I have no idea what kind of agent could make such elementary mistakes.

[u/pron98 → u/TomKavees]

The JVM was designed to be faster than native code in a large class of applications by exploiting optimisations that are unavailable to native code. If both Rust and Java work well, what you'd expect to see is Rust being consistently faster in some applications and Java consistently faster in others.

There is no reason to think that "AOT-compiled native code" has a universal performance advantage. After all, the JVM was invented to, among other things, reduce the overheads that low-level languages typically suffer from and that can be significant in many programs.

[u/brunocborges → u/TomKavees]

Eventually Java applications do get compiled down to native code. So, it is a native code (Java) vs native code (Rust) comparison.

[u/aoeudhtns → u/TomKavees]

There are many workloads where native is consistently slower. Particularly around total throughput. With native you do the memory management (typically) on each request because the GC is gone, so you pay that cost spread out across everything vs. a latency spike here and there. And the C2 compiler is very, very good so it's not strictly true that AOT native is faster than normal Java.

One often finds themselves in situations where you have to decide what's the most important: startup, latency (in terms of maximums or deviation), and total memory consumption; or total throughput. That is typically the tradeoff I see in my own comparisons of native vs. regular Java.

[u/javaprof]

It's not apples to apples comparison, Java code adopted to be faster in benchmark https://github.com/kgonia/once-campfire-java

Even tho:

Startup	Rust ~0.24 s; JVM ~2 s, then warmup
Peak memory with default settings	Rust 132 MiB; JVM ~1.3 GiB

[u/pron98 → u/javaprof]

First, the "default setting" in Java for heap consumption is pretty arbitrary, and shouldn't be used at all although that will change in JDK 28 with automatic heap sizing for ZGC. Second, there's a lot of misunderstanding around RAM usage and how it matters. In many situations, using 5x or even 10x more RAM is the more efficient use of resources even if the result is a 5% reduction in CPU. That's because RAM and CPU are not independent resources, and because of that it's almost always more efficient to increase the use of the less constrained resource to reduce the use of the more constrained one.

[u/doobiesteintortoise → u/javaprof]

This is a better comparison, IMO, but similar methodology: https://www.wecodefire.com/p/kotlin-keeps-up-with-rust

(It's not mine, I haven't done anything like this and I find the results interesting but that's about it.)

[u/snuderl]

Aot here should really just be helping startup/warmup and reach the same end performance as without aot right?

[u/ThaJedi → u/snuderl]

To what point exactly you refer? It exactly does that so what I stated that this is unclear?

[u/[deleted] → u/snuderl]

[u/BenchEmbarrassed7316]

DHH is porting campfire to Rust.

You don't need to read any further. In case you haven't seen an example of his "code" - here it is: https://www.reddit.com/r/rustjerk/comments/1wusj30/i_think_this_guy_is_lying_ai_isnt_capable_of/ This is not a typical vide-code. I don’t know how one could get AI to generate code this bad. Perhaps it’s using skills like "Make moar mistakes", "Use every known anti-pattern" and "Generate the slowest code as possible". Even if you simply generated a similar Java application using AI, it is most likely written much better than whatever this clown is producing.

Therefore, this benchmark makes no sense at all.

edit: I’ve written some explanation here: https://www.reddit.com/r/java/comments/1x9s75g/comment/phud9np/

[u/crusoe → u/BenchEmbarrassed7316]

He must be using some cheap Chinese model or forcing everything through Haiku...

[u/chic_luke → u/BenchEmbarrassed7316]

A person in that thread brings up a great point about mental ownership. That same point is the reason why, unfortunately, you can't just effortlessly fork an open source project gone rogue, especially if it's high in complexity and already maintained by a bunch of people, none of which can be convinced to switch to your hard fork.

Even if vibe coding works when you judge the runtime results - which is pretty clear to everybody who's been keeping up to date that it does, and quite well - the mental ownership dilemma is very much here to stay.

Proponents of KISS software, even if they have taken a very extreme stance, have warned us of this risk decades before AI. It just happens to be relevant now.

[u/Plenty_Childhood_294]

Let me leave this here ;) https://github.com/franz1981/campfire-quarkus-bench

[u/ThaJedi → u/Plenty_Childhood_294]

I did small run on my machine and Java version is very comparable to Kotlin except memory usage. I'll drop numbers after testing on fresh machine.

[u/Prior-Equal2657]

Unfortunately it seconds my experiments.

Java is fine with performance and throughput, but RAM usage is simply incomparable.

As much as I like java/kotlin i'm looking more and more towards rust because of it's memory usage. With some wild optimizations I was able to get to like 3-4x RAM of rust, but that's not the code I want to see in production.

Anyway, at least I hope that electron apps will eventually vanish. That's easy 20x RAM of similar rust apps on desktop.

[u/oweiler]

JEP 544 only improves time to peak performance, not performance in general.

[u/javaprof]

Some numbers drop without code?

[u/zerg000000]

great job! Java! Java did very good job at optimizing. It is just hard to beat Java at the things it good at. But Java still have some area that will never as good as low level language/static language. 1) memory usage, GC is both a bless and a curse, many tiny objects would use a lot of memory. 2) hot loop can get pretty close to low level language but never have the chance to finetune further. 3) It is reference based, locality will be less strong, which hurt the chance of using SIMD more widely and the cpu cache.

[u/Brutus5000]

What is campfire? 

  Campfire is a web-based group chat application built on Ruby on Rails that allows users to communicate in multiple rooms, send direct messages, and share files. It is designed for self-hosting, giving users control over their data and customization options.

Doesn't look like something that requires heavy benchmarks?

[u/benevanstech]

Quick, rig the benchmarks. We don't want DHH bringing his awful toxic bullshit over into our community.