When the fractional part of a float fixes your shader
summary
A visual stutter in a Voronoi-diagram shader was caused by a compiler optimization bug in DirectX 11's FXC compiler. The issue occurred when the compiler incorrectly optimized away the fractional part of a float during the calculation of noise centers, but only when specific non-decimal literal values were used. This led to a loss of precision that manifested as snapping animations on specific hardware and OS configurations.
key points
- The stutter only appeared on Windows machines using the ANGLE layer to translate GLSL to HLSL for DirectX 11 rendering, while other GPUs and APIs like Vulkan remained unaffected.
- Replacing the 'fract()' function with 'x - floor(x)' or changing a literal float from 398.0 to 398.1 bypassed the bug by preventing the compiler from incorrectly truncating the value.
- Disassembly via RenderDoc revealed that the O3 optimization level in the FXC compiler was aggressively reassigning registers to their own floor values, effectively deleting the fractional data required for smooth animation.
who should read it
Graphics programmers and shader developers who are interested in GPU debugging, compiler optimizations, and the intricacies of the WebGL rendering pipeline.
discussion positive
consensus
Commenters generally agree that shader programming is unintuitive and prone to subtle, platform-specific bugs due to inconsistencies in floating-point operations across different GPUs. While LLMs are praised for their ability to generate working GLSL code, users warn that relying on them prevents a true understanding of the underlying logic. There is a shared sentiment that graphics programming often prioritizes visual results over strict mathematical accuracy, making debugging a tedious process of trial and error.
top insight
The distinction between `fract(x)` and `x - floor(x)` is critical because `fract(x)` behaves differently with negative values, often leading to logic errors in lerp variables and visual artifacts. This insight clarifies that common shader bugs often stem from these small mathematical discrepancies rather than high-level architectural failures. It emphasizes the need for developers to be mindful of specific function behaviors when creating lattices or coordinate systems.
Generated by an LLM from the article and thread. Check the sources before relying on it.