Cognition launches new SWE-2 model, Rivaling Fable 5.1 and GPT-Astra
summary
SWE-2 is a new coding model that optimizes the trade-off between performance and cost by scaling reinforcement learning to a multi-trillion parameter regime. It utilizes a principled cost-penalized reward function to train multiple effort levels simultaneously, ensuring the model remains intelligent while reducing redundant operations. This approach allows the model to achieve near-frontier performance while being significantly cheaper and more focused in its problem-solving process.
key points
- The model employs Pareto-informed cost penalties during reinforcement learning to directly optimize the balance between solve rates and operational costs. This ensures that increasing the model's reasoning effort actually leads to better results rather than just higher spending.
- Efficiency is improved through focused exploration, where the model uses better engineering judgment to identify relevant codebase sections and start implementation sooner. This reduces the number of steps and turns required to solve a task compared to previous versions.
- Technical stability is enhanced using a length-weighted reward baseline and an online draft model for speculative decoding. These improvements reduce gradient variance and increase decoding throughput, allowing for more stable training of a very large model.
who should read it
AI researchers and software engineers interested in the intersection of large language models, reinforcement learning, and automated software engineering.
discussion skeptical
consensus
Commenters generally agree that 'benchmaxxing'—the practice of optimizing models specifically to score high on benchmarks—is an industry-wide norm and often renders older benchmarks like Terminal Bench 2.1 obsolete. There is a strong sentiment that benchmarks are frequently used as marketing tools rather than accurate measures of general capability, leading to a significant performance drop when models are tested on newer, out-of-sample benchmarks like Terminal Bench 4. While some users report actual improvements in model utility, many remain skeptical of the company's claims and valuation given this pattern.
top insight
One commenter argues that benchmaxxing is an inevitable failure mode of any learning system, comparing it to biological evolution where organisms optimize for the single metric of reproductive success. This perspective suggests that overfitting to a target is not necessarily a malicious choice by developers, but a fundamental characteristic of how intelligence adapts to a given reward signal.
Generated by an LLM from the article and thread. Check the sources before relying on it.