Concise-9B: Removing the Slack from a Reasoning Model's Thinking
Alexander Boesgaard Lorup
Carefully measured, honest empirical report with a useful slack decomposition, but the method is explicitly not new and the gains are model-dependent.
Abstract
Reasoning models answer better when they think longer, but a thinking trace keeps going well after its answer has formed. We measure this slack in Qwen3.5-9B and remove most of it with concise self-training: two rounds of on-policy rejection-sampling fine-tuning that keep the shortest correct samples, plus a small share of answer-synthesis targets, giving Concise-9B. On 8,983 held-out problems (competition and hard mathematics, SuperGPQA, a five-area confirmation set and MMLU-Pro), Concise-9B generates 0.66x the tokens of Qwen3.5-9B at a 32K thinking budget, with accuracy equivalent within +/-1 point (-0.20 [-0.77, +0.36] points). Forcing each trace on the hard and confirmation sets to answer every 2,048 thinking tokens, we find that 54-64% of Qwen3.5-9B's thinking tokens come after the point from which every forced answer equals the final one (41-48% for Concise-9B). Stopping there would keep every answer at 0.35-0.44x the base model's tokens, but finding that point takes hindsight: stopping when three consecutive forced answers agree costs the base model 2.4-9.2 points. Concise self-training removes slack without that cost: 86% [83, 90] of the thinking tokens it saves come after the answer has formed. It has a limit: with traces continued to 80K tokens, Concise-9B solves 25 fewer of 1,276 hard problems (-25 [-49, -1]), mostly in competition mathematics. The measurement also yields a cheap commit signal for adaptive sampling that matched two-sample escalation within +/-1 point at 0.67x its tokens in a pre-registered test. Results on Qwen3.5-4B, on which the recipe was developed, and a replication on Olmo-3-7B-Think suggest that the saving depends on how verbose the base model is.
AI Impact Assessments
(1 model)Core Contribution
The paper is a technical report that does two things for Qwen3.5-9B. (1) It builds Concise-9B with two rounds of on-policy rejection-sampling fine-tuning. Each round keeps the two shortest correct samples per problem and adds about 9% answer-synthesis targets. (2) It measures "slack": it forces an answer every 2,048 thinking tokens and finds the point after which every forced answer equals the final one. The authors say plainly that neither the training recipe nor the stopping rule is new. They credit Munkhbat et al., Yang et al., Kimi k1.5 and Liu & Wang. The contribution is the measurement and the large-scale, tightly bounded comparison, which gives four findings:
Methodological Rigor
Several design choices are strong:
There are also gaps:
Potential Impact
The practical result is a roughly one-third token reduction at equivalent accuracy on a popular open model. That is useful but not novel as a technique. The more durable value is the framing of slack and the hindsight-versus-online gap. It shows that answer-convergence early stopping, which several recent papers promote, loses substantial accuracy on hard problems on a native thinking model. It also shows that training implicitly "knows" which tokens are slack. The self-agreement commit signal is a small, reusable idea for adaptive-sampling pipelines. The Olmo replication, with only a 1–4% token reduction, shows the benefit depends on base-model verbosity. That limits generality but is an honest and informative finding.
Timeliness & Relevance
Efficient reasoning and overthinking are very active topics, and inference cost for thinking models is a real bottleneck. Qwen3.5 is a current open model family. However, the field is crowded with RL-based length-control methods, early-exit methods and shortest-correct SFT. A paper that explicitly claims no new algorithm will be cited mostly as a measurement reference, not as a method.
Strengths
Limitations
Overall this is a solid, careful empirical report with modest novelty and moderate impact. It is useful for practitioners and for efficient-reasoning researchers who want a measurement baseline, but it is unlikely to become a foundational reference.
Generated Nov 3, 2026
Comparison History (0)
Carefully measured, honest empirical report with a useful slack decomposition, but the method is explicitly not new and the gains are model-dependent.