Replacing a Rust Enum with a 64-Bit Word Made My Interpreter 17% Faster

summary

Replacing a 16-byte Rust tagged enum with a 64-bit tagged word representation significantly improves interpreter performance by reducing memory traffic and allowing values to fit into CPU registers. This approach leverages low-bit tagging to store integers, floats, and pointers within a single word, minimizing the overhead of memory alignment and stack spilling. Despite the need for occasional heap boxing for overflow values, the gains in cache efficiency and streamlined machine code outweigh the costs. Ultimately, this change results in a consistent speedup across various benchmarks, including floating-point heavy tasks.

key points

1. Low-bit tagging enables the storage of 62-bit integers, floating-point values, and pointers in a 64-bit word by stealing unused bits from memory addresses and integer ranges. This reduces the memory footprint of each value, leading to better cache utilization and fewer memory accesses during execution.
2. Replacing the Rust enum with a u64 newtype prevents the compiler from generating inefficient code characterized by excessive stack spilling and disjoint basic blocks. The resulting machine code is more linear and compact, allowing the CPU to handle value dispatch and arithmetic operations more efficiently.
3. The implementation of a self-tagging scheme for floating-point numbers allows most doubles to be stored directly in the tagged word without losing precision. While this requires more instructions to unbox and re-box than simple integers, it is still faster than the previous enum-based approach due to the reduction in memory operations.

who should read it

VM engineers, language designers, and Rust developers interested in low-level memory optimization and interpreter performance.

discussion positive