I turned Jev into a (lousy) chatbot
summary
Jevchat is an experimental tool that converts a decision-based AI model called Jev into a generative chat model by iteratively asking it to predict the next symbol in a sequence. The system works by providing the model with a set of options and having it return a probability distribution for each, which is then sampled to build a response. While the cost is impractical for real-world use, the project serves as a technical exploration of sampling strategies and alphabet management.
key points
- The system implements various sampling strategies, including a simple choice method, a bisection method for efficiency, and a bucketed approach to handle larger vocabularies.
- Switching from symbol-based options to hypothesis-based options significantly improves accuracy by allowing the model to rank completed strings rather than individual characters.
- The the system supports configurable alphabets ranging from basic ASCII to larger BPE token sets, with beam search capabilities to maintain multiple candidate responses.
who should read it
Software engineers and AI researchers interested in LLM sampling techniques, tokenization, and the creative application of decision models.
discussion positive
consensus
Commenters generally view Jev as an interesting but potentially narrow tool, with some questioning if its 'secret sauce' is merely well-calibrated probabilities or fast parallel decoding. There is a shared nostalgia for the chaotic, 'demented' output of early LLMs compared to today's more sanitized models. A significant portion of the discussion centers on the rapid acceleration of the 'idea-to-output' cycle, where common implementation ideas are now realized by multiple people almost simultaneously.
top insight
The rapid collapse of the time between a technical idea and its public implementation is fundamentally changing the nature of hobbyist exploration. One user notes that the drive to build for 'fun and learning' is being eclipsed by a race to output, where ideas are harvested and implemented so quickly that the individual satisfaction of discovery is diminished.
Generated by an LLM from the article and thread. Check the sources before relying on it.