Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia
Janus is a single Go binary that runs GGUF models locally via llama.cpp and Vulkan, exposing an OpenAI-compatible API without requiring Python or Docker.
summary is enough The README cleanly outlines the architecture and setup steps, making a deep read unnecessary unless you plan to use or contribute to the project.
summary
Janus provides a lightweight, dependency-free mechanism to serve GGUF models on local GPUs or CPUs using a single Go executable. It wraps llama.cpp via Vulkan bindings, offering an OpenAI-compatible API, model hot-swapping, and built-in support for chat templates and reasoning splits. Commenters point out that performance is tied directly to upstream llama.cpp since Janus acts as an FFI wrapper.
key points
- Operates as a standalone Go binary using llama.cpp via Vulkan or CPU backends with zero Python dependencies.
- Exposes standard OpenAI-compatible endpoints like /v1/chat/completions with streaming and hot-swapping support.
- Automatically detects GGUF chat templates and parses thinking model reasoning splits.
who should read it
Developers looking to run local LLMs with minimal dependencies or integrate GGUF inference into Go workflows.
discussion positive
consensus
Commenters quickly established that Janus functions as a thin wrapper around libllama, meaning its performance characteristics mirror upstream llama.cpp rather than specialized high-throughput engines.
pushback
A few Intel hardware users expressed mild disappointment that it relies on Vulkan rather than Intel-specific frameworks like OpenVINO or SYCL.
Since this is basically a wrapper around libllama.so, I would assume that the performance is roughly the same as llama.cpp upstream.
Clarifies the core technical implementation, noting that performance will match upstream llama.cpp because it is essentially a wrapper around libllama.so.
Generated by an LLM from the article and thread. Check the sources before relying on it.