Beating the compiler (2024)
Matt Keeter's hand-written AArch64 assembly interpreter for the Uxn CPU outperforms an optimized Rust baseline by 30% through register allocation and threaded code.
read it The piece offers a concrete look at where hand-written assembly still beats modern compilers in interpreter loops, backed by detailed benchmarks and ablation data.
summary
The author investigates whether hand-written assembly can outperform a compiler-generated interpreter loop for the stack-based Uxn CPU. By moving critical state into specific registers and replacing a central dispatch loop with threaded code, the resulting assembly interpreter runs 30% faster than optimized Rust. Ablation testing reveals that central dispatch eliminates most of this advantage, highlighting the cost of unpredictable branch targets. The post demonstrates how language abstractions prevent expressing certain low-level optimization patterns like direct threaded code.
key points
- Hand-written AArch64 assembly achieved a 30% speedup over an optimized Rust baseline for Fibonacci and Mandelbrot workloads on Uxn.
- Using threaded code where each opcode ends in a direct jump to the next eliminated costly indirect branch mispredictions.
- Keeping all vital VM state in dedicated CPU registers eliminated superfluous memory loads and stores.
who should read it
Systems programmers, compiler engineers, and developers interested in low-level optimization and interpreter design.
discussion positive
consensus
Commenters agree that while compilers are exceptionally smart, hand-written assembly can still win where language abstractions prevent expressing specific patterns like threaded code.
pushback
A minority notes the heavy maintenance burden of hand-crafted assembly as hardware and compilers evolve over time.
There are literally thousands of compiler engineers who pore over the assembly a compiler generates, and then finds ways to make it better. I get paid to figure out where the compiler can do better, and often a step in that is to hand-code my own replacement. I then teach the compiler to do that.
But even beyond that, the compiler can't make certain assumptions that an assembly writer can. Such as whether a callee-saved register really does need to be saved in some particular routine. Or even pushing an extra parameter in unusual cases.
So it is entirely possible to beat the compiler, it's …
A working compiler engineer explains why compilers can't always make certain assumptions about register saving or unusual parameters, validating manual optimization.
Generated by an LLM from the article and thread. Check the sources before relying on it.