The question of how much computation to spend on any given input has become one of the more practically consequential problems in deploying large reasoning models. Systems like Qwen3 and DeepSeek-R1 generate extended chains of intermediate reasoning before committing to a final answer, and the right token budget varies enormously across instances. Spending a full budget on a trivially easy question wastes resources; cutting off a hard question before it self-corrects hurts accuracy. This paper, available on arXiv, asks a specific and well-scoped question: when does a learned stopping rule actually improve over simpler scalar signals like confidence or answer stability?
That framing matters. A learned stopper is not free. It requires training data, threshold selection, and an inference infrastructure that can pause generation, branch a probe answer, and resume the original reasoning state. If a single confidence threshold achieves the same result, the complexity is unjustified. The authors take this cost seriously throughout, which distinguishes the work from much of the early-exit literature that reports accuracy-compute curves without accounting for probe overhead.
What LearnStop Does
LearnStop is a logistic classifier trained over prefix-observable features computed at fixed token-budget checkpoints. At each checkpoint, the system forces a short answer from the current reasoning prefix and extracts features including answer confidence, output entropy, prefix vote share (agreement across sampled completions), answer stability across checkpoints, and backtracking-marker density in the reasoning trace. Crucially, it avoids hidden states entirely. This makes it portable across model families without requiring access to internal activations, though it still requires an interactive decoder capable of prefix branching.
The deliberate simplicity of the classifier is a methodological choice worth appreciating. The authors test gradient boosting and MLPs, which marginally outperform logistic regression on GSM8K-32B, but logistic regression wins on MATH-500 and remains competitive elsewhere. Simpler models are also easier to calibrate and less prone to overfitting on the relatively small training sets available from reasoning traces. Defaulting to logistic regression is a reasonable practical choice.
The study spans 18 task-model combinations: Qwen3-8B and Qwen3-32B as primary models, with DeepSeek-R1 distillations for cross-family checks, evaluated on GSM8K, MATH-500, MMLU-Pro, GPQA-Diamond, and AIME-90. The budget grid runs from 128 to 1536 tokens for most tasks, extending to 6144 for AIME.
Main Findings
The central result is task-dependent, and the authors present this as a finding rather than a limitation to minimise. On free-form math, learned stopping improves the accuracy-cost frontier meaningfully. On GSM8K with Qwen3-32B, the peak adapt gain reaches +0.157 over fixed-budget baselines, and the paired gain over the strongest scalar baseline is +0.028. On MATH-500 the gains are positive but smaller. On multiple-choice tasks like MMLU-Pro and GPQA, scalar confidence or entropy exits are competitive or stronger than the learned policy. On AIME-90, which is both small and very hard, results are unstable and the authors wisely treat it as a stress test rather than a source of positive claims.
Key findings summarised:
- Learned multi-feature stopping improves the fixed-budget frontier on free-form math, particularly GSM8K and MATH-500 with Qwen3 models.
- On multiple-choice tasks, confidence and entropy alone are already near-optimal stopping signals; adding learned features provides little benefit.
- No single feature family explains all math gains: on GSM8K-32B, entropy-only achieves +0.136, confidence-only +0.088, stability-only +0.093, and the full feature set +0.157. The features are partially complementary.
- Removing length and end-of-reasoning features changes peak gain by less than 0.007, confirming the gains are not artefacts of those potentially confounded signals.
- Probe overhead can erase compute savings under black-box repeated-prefill accounting. Prefix caching is essentially required for the method to be cost-positive in practice.
The trajectory decomposition analysis gives the clearest mechanistic explanation. Learned stopping is most useful when a substantial fraction of questions become correct before full budget but do not exhibit a single reliable scalar signal of that correctness. When confidence alone tracks correctness well, there is nothing left for the multi-feature classifier to learn.
Cost Accounting and Calibration
One of the more valuable contributions is the deployment cost analysis. The paper considers three regimes: KV-fork (branching the KV cache to run a probe without discarding the reasoning prefix), prefix-cache (recomputing the prefix but caching it), and black-box API (full repeated prefill with no state reuse). H100 wall-clock profiles are provided for each. The conclusion is stark: under black-box accounting, probe overhead frequently exceeds the token savings from early stopping. This is not a minor caveat; it is a practical constraint that eliminates a large fraction of deployment scenarios where the method would otherwise appear attractive.
The risk calibration framework is also worth attention. The authors define lost-correct risk as the probability that early stopping flips a full-thinking correct answer to an incorrect one, and they calibrate thresholds using a finite-grid upper confidence bound on this quantity. This gives a principled way to select operating points with bounded accuracy loss, rather than selecting purely on the accuracy-cost frontier. Validation-selected operating points are more conservative than peak frontier gains, and the paper reports both, which is the right approach.
Limitations and Broader Implications
Several limitations deserve emphasis. The peak adapt gain is a frontier summary computed post hoc; it is not directly deployable. Validation-selected and risk-calibrated points are substantially more conservative and should be the reference for practical comparisons. The PUMA and Thought Calibration comparisons are approximations because the original artifacts require hidden states or are not fully reproducible from available checkpoints. AIME-90 and GPQA-Diamond have small test sets, so variance is high and negative results there cannot be interpreted as precise method rankings.
The requirement for prefix reuse infrastructure is a real barrier. Most production serving stacks do not expose the kind of interactive branching that KV-fork requires. This limits the method to settings where the serving infrastructure can be modified, which is a non-trivial constraint for many practitioners.
The broader implication is a reframing of early exit research. Rather than asking which stopping method wins universally, the question should be which stopping signal best captures the trajectory structure of the target task given the actual deployment cost model. For free-form math with partial prefix reuse, learned multi-feature stopping is a reasonable component. For multiple-choice tasks or black-box APIs, a well-tuned confidence threshold is probably sufficient and much cheaper to operate. This is a useful and honest conclusion, and the paper earns it through careful empirical work rather than asserting it from theory alone.
The code and processed outputs are available at github.com/dongzhe1/llm-think-stop, and the full paper is on arXiv (2606.30852).