MicroLLM Lab – Try 7 tiny LLM's in the browser
MicroLLM Lab lets users run, benchmark, and chat with seven quantized small language models entirely in the browser using WebGPU.
summary is enough The browser demo is fun to test briefly, but the tiny models are mostly incoherent and the value lies in seeing WebGPU inference firsthand.
summary
MicroLLM Lab is a zero-server web application that uses WebGPU and 4-bit quantization to run small language models ranging from 25M to 360M parameters directly in the browser. The experiment allows users to chat with models locally, cache them via IndexedDB, and run performance benchmarks. While the local execution speed is impressive, commenters note that models of this size struggle significantly with basic reasoning, math, and practical tasks like cooking recipes. The thread also features lively discussion on browser WebGPU support, UI design, and links to similar experimental local-AI projects.
key points
- Models use Q4 quantization to shrink parameters down to 50-84 MB, enabling fast local execution via WebGPU.
- Tiny models (25M-360M) achieve high token-per-second speeds locally but frequently fail basic logic, math, and factual queries.
- Firefox on Linux requires manual workarounds or WASM fallbacks because WebGPU is not enabled by default.
who should read it
Front-end and AI engineers interested in local browser-based machine learning and WebGPU capabilities.
discussion positive
consensus
Commenters are impressed by the raw speed of in-browser WebGPU inference, but note that the models themselves are mostly useless for practical tasks due to severe hallucinations and logic failures.
pushback
A few users criticize the dense, AI-generated-looking user interface for burying the actual interactive chat interface below walls of text.
Can someone help me understand whether these models are supposed to be good enough to be useful? I am asking these for a breakfast recipe and most of them are repeating text and giving me strange combinations of like chicken & parmesan or milk and like 8 cups of cheese. I really would love to use MicroLLMs and have been eagerly awaiting the day they could be useful, or at least run local LLMs for that matter, but these don't seem useful.
They don't even consistently pass the benchmarks included in the site, so what are they good for?
Edit: or is the purpose to just showcase that these small …
It provides a realistic reality check on the practical utility of sub-billion parameter models by testing them on a concrete, everyday task.
Generated by an LLM from the article and thread. Check the sources before relying on it.