Fable 5 – Median thinking declined in August
summary
Anthropic's Fable 5 model experienced a noticeable decline in performance and reasoning quality after becoming permanently available in subscription plans. This drop is not a result of the model being 'nerfed' in terms of its core weights, but rather a change in the inference regime used to serve the model to users. By reducing the number of thinking tokens generated during inference, the provider effectively limits the model's ability to reason through complex problems.
key points
- Comparing data from July and August reveals a significant decrease in the number of thinking tokens provided to the model during inference.
- The model's performance fluctuates over multi-day episodes, often aligning with product announcements and releases, suggesting active management of inference resources.
- Users should shift their focus from questioning if a model's intelligence is fundamentally degraded to questioning which specific inference regime they are being served.
who should read it
AI researchers, power users of LLMs, and developers who rely on frontier models for complex reasoning tasks.
discussion positive
consensus
Commenters generally agree that frontier LLM performance is unstable and prone to 'rot' or degradation over time, often attributed to cost-cutting or revenue optimization by providers. This frustration drives a growing interest in self-hosting open-weight models like Qwen and GLM to ensure predictable, constant quality and reliability. While some argue that open models still lag behind the frontier in raw capability, many find the trade-off acceptable for the sake of stability and control.
top insight
The perceived degradation of closed models is not just about the model weights, but the opaque, real-time adjustments to the inference harness, including load balancing and software settings tuned for revenue optimization. This makes it a controlled experiment impossible for external users, as the experience is managed by A/B testing and dynamic resource allocation rather than a static model version.
Generated by an LLM from the article and thread. Check the sources before relying on it.