Jeeves. Reasoning improves Jev-like decision models
Jeeves is an open-source 9B Qwen-based decision model that adds test-time reasoning via SFT and CISPO to improve Jev-like classification accuracy.
summary is enough Skim this repository if you are evaluating open-source decision models, as it details how adding test-time compute trades latency for higher accuracy.
summary
Jeeves adapts the Qwen3.5-9B architecture with LoRA and a pointer head to function as a Jev-like decision model with test-time reasoning capabilities. Trained using Supervised Fine-Tuning and CISPO reinforcement learning, it achieves higher accuracy on complex classification benchmarks than baseline Jev models. However, enabling full reasoning chains increases median latency to over three seconds, prompting the inclusion of a block-4 diffusion drafter and configurable thinking thresholds. The repository provides full training code, evaluation harnesses, and a drop-in Python SDK compatible with Jev APIs.
key points
- Jeeves builds on Qwen3.5-9B using SFT and CISPO to enable test-time reasoning for structured decision-making tasks.
- It outperforms baseline Jev and Kev-9B models on out-of-domain and JevBench public tiers, scoring 0.935 overall on JevBench.
- Full reasoning increases median latency to 3.3 seconds on an H100, which can be mitigated via token caps, no-think thresholds, or an integrated diffusion drafter.
who should read it
ML engineers and developers looking for open-source alternatives to proprietary decision and classification models.
discussion mixed
consensus
Commenters debate whether adding slow reasoning defeats the core value proposition of Jev-style models, which are valued for sub-second, cheap inference.
pushback
A minority point out that the trade-off is configurable via thresholds and caps, making it useful for selectively handling harder moderation or classification tasks.
If the model does autoregressive reasoning before the decision, doesn't that give up much of what a Jev-style model buys you (a single forward pass, cheap calibrated probabilities)? Or is the point mainly to keep the typed output and probability interface while getting better accuracy on harder cases?
It clearly frames the central architectural tension: whether autoregressive reasoning undermines the core benefit of single-pass, cheap Jev models.
Generated by an LLM from the article and thread. Check the sources before relying on it.