Cognition's SWE-2 achieves 92.8 on Terminal-Bench 2.1
summary
Cognition has released SWE-2, a massive Mixture-of-Experts model based on Kimi K3 that utilizes advanced RL post-training to significantly improve agentic coding capabilities. By scaling RL into the multi-trillion parameter regime, the model achieves high efficiency and lower costs compared to frontier models like GPT-6 Astra and Claude Fable 5.1. While it excels in short-to-medium horizon tasks, it still struggles with the long-horizon agentic work required for complex software engineering. The system's performance is boosted by a specific serving stack including NVFP4 and FP8 kernels and a draft model for faster inference.
key points
- SWE-2 uses a Multi-trillion parameter MoE architecture with 104B active parameters per token, leveraging NVFP4 and FP8 kernels to optimize inference speed and performance.
- The model demonstrates superior cost-efficiency, claiming 64% lower costs than Claude Fable 5.1 at similar performance levels on FrontierCode, while requiring significantly fewer turns to reach a solution.
- Despite strong performance on basic benchmarks, a significant gap remains in long-horizon agentic work, as evidenced by its lower scores on Terminal-Bench 4.0 compared to competitors.
who should read it
Software engineers and AI researchers interested in agentic coding models, LLM optimization, and the competitive landscape of frontier coding assistants.
discussion skeptical
consensus
Commenters generally distrust current AI benchmarks, viewing them as saturated or contaminated by training data. Many argue that specific high scores on older benchmarks like Terminal Bench 2 are irrelevant compared to newer metrics. There is a strong skepticism regarding the actual real-world utility of these models versus their benchmark performance.
top insight
The debate over 'model nerfing' suggests a pattern where providers may secretly quantize models or route requests to cheaper versions after initial hype to reduce compute costs. While some dismiss this as a conspiracy theory, others argue it creates a profit incentive by forcing users to consume more tokens to achieve the same result.
Generated by an LLM from the article and thread. Check the sources before relying on it.