Show HN: Npunlock – Run custom C kernels for Intel NPUs
summary
Intel NPUs contain programmable SHAVE cores that are typically hidden by high-level graph programming, limiting flexibility. npunlock provides a way to bypass these restrictions by reconstructing the missing path from custom C code to runnable NPU kernels. This allows developers to execute custom, high-performance operations like GELU on the NPU while still utilizing Intel's existing drivers and compilers for graph management.
key points
- The tool allows developers to write kernels in C and compile them into ACT-SHAVE machine code, enabling the following of operations not natively supported by the standard NPU software stack.
- It supports static dense FP16 unary and two-input kernels, as well as verified FP32 unary paths, enabling nonlinear math functions like tanhf and GELU on the NPU hardware.
- The system integrates with Python and C APIs, using a specific toolchain called MoviTools for compilation, allowing for easy verification against NumPy references on Windows x64 with Meteor Lake hardware.
who should read it
Low-level hardware developers and AI researchers who want to implement custom operations on Intel NPUs and bypass the standard high-level software constraints.
discussion positive
consensus
Users and the author agree that Intel's NPU compiler is unpredictable and often performs suboptimal or silent precision lowering. There is a shared sentiment that developers should have bare-metal access to NPU hardware to bypass the limitations of OpenVINO. The community views the project as a vital tool for implementing custom kernels and gaining hardware control.
top insight
The project's primary value lies in enabling the creation of 'mega kernels' that fuse multiple small kernels together. This approach reduces SHAVE invocations, which potentially improves performance beyond what standard Intel tools allow. It also allows for the high-precision FP32 implementations of operators that are otherwise silently downgraded by the compiler.
Generated by an LLM from the article and thread. Check the sources before relying on it.