Needed 1+1, built a functional programming language
A developer trying to evaluate a simple arithmetic expression in C ended up building a custom graph reduction language complete with an arena allocator, chunk memory management, and a garbage collector.
read it The entertaining postmortem offers a delightful deep dive into building memory management and garbage collection from scratch in C.
summary
A simple data structures homework assignment to evaluate an arithmetic expression snowballed into building a functional programming language in C from scratch. The author details the architectural evolution, including shifting from specific math nodes to generic functions, implementing an arena allocator, upgrading to a chunked allocator to prevent pointer invalidation, and finally writing a mark-and-sweep garbage collector. This GC drastically reduced RAM usage for large recursive inputs like fib(40) from 12 gigabytes to 1.7 megabytes. The piece highlights the classic rabbit holes of systems programming and yak-shaving.
key points
- Representing all operations as generic functions rather than individual types unified variables and closures under a single graph structure.
- Switching from a single growing arena to a chunked linked-list allocator avoided pointer invalidation during memory growth.
- Implementing a mark-and-sweep garbage collector dropped RAM usage for computing fib(40) from 12.29 GB to 1.72 MB.
who should read it
Systems programmers and hobbyist language implementers who enjoy tales of rabbit-hole engineering.
discussion warm
consensus
Commenters heavily relate to the classic yak-shaving rabbit hole of simple projects turning into massive engineering efforts, referencing Greenspun's tenth rule and sharing similar war stories.
pushback
Some technical commenters question the necessity of custom memory allocators over heavily optimized modern libraries like mimalloc or tcmalloc.
It's tempting to go down the rabbit hole of getting creative with memory management as the author does, and maybe worthwhile for educational purposes, but has anyone here observed any noticeable performance gain from it? My impression is that libraries like mimalloc and tcmalloc are painstakingly optimized by specialists and unlikely to benefit from any casual embellishments beyond using the supported malloc and free functions for everything. What does the HN braintrust recommend?
It prompts a pragmatic discussion on whether writing custom memory management pays off compared to battle-tested modern allocators.
Generated by an LLM from the article and thread. Check the sources before relying on it.