← Home

Quantifying Redundancy in LLM Reasoning Chains

By James Trappett · 27 May 2026

4 min read

Anyone who has watched a frontier reasoning model work through a simple algebra problem has noticed the phenomenon: dozens of steps of reformulation, second-guessing, and circular self-reflection before arriving at an answer that required perhaps three lines of working. This paper, arXiv:2605.23926, is the first systematic attempt to measure exactly how much of that deliberation is unnecessary, and to explain from first principles why it occurs at all. The answers are striking on both counts.

What the Paper Contributes

The authors make three distinct contributions that build on each other cleanly.

First, they formalise a definition of reasoning redundancy that is grounded in the model's own behaviour rather than human judgement or proxy metrics. The redundancy of a correct trace is defined as the largest fraction of trailing segmented steps that can be removed while the model, forced to terminate and emit an answer immediately, still produces the correct result. This is measured by progressive truncation: strip steps from the end, one at a time, and find the minimal prefix sufficient for correctness. The critical prefix k* is the smallest number of leading steps from which the model can still recover the right answer.

Second, they apply this definition at scale across four frontier reasoning models (DeepSeek-R1, QwQ-32B, R1-Distill-Qwen-7B, and Qwen3-30B-Thinking), two benchmarks (GSM8K and MATH-500), and two independent judges, yielding 1,880 correct traces for analysis. This is not an anecdote study.

Third, they prove a theoretical result explaining why redundancy is so high: under any length-agnostic outcome reward (the standard training signal for all current reasoning models), no finite expected stopping time is optimal. Over-thinking is not a bug in any particular recipe; it is the optimal behaviour under the reward that every known recipe uses.

Methodology

The experimental setup is careful in several respects worth noting. Traces are segmented into discrete steps, and redundancy is measured at step level rather than token level, which avoids conflating verbosity within a step with structural redundancy across steps. Three independent traces are drawn per problem at temperature 0.7, giving a within-problem variance estimate as well as a mean. An external judge (GPT-4o-mini) is used as a robustness check alongside the model's own decoding, and the gap between the two measures (reported as gap in the main table) is informative: it ranges from about 10 to 60 percentage points, reflecting how much easier it is for the model itself to complete a partial trace than for an external system to reconstruct the answer from it.

Bootstrap confidence intervals are computed with 10,000 resamples at the problem level rather than the trace level, which correctly accounts for within-problem correlation. All eight (model, benchmark) conditions have CIs narrower than ±4 points, so the headline numbers are statistically solid.

The theoretical framework models reasoning as a sequential decision process with three action types: Advance (substantive progress toward the solution, succeeding with probability p), Idle (non-advancing content such as verification or reformulation), and Stop (terminating and emitting the answer). The outcome reward is 1 if enough Advance steps have succeeded, 0 otherwise. The key theorem shows that under this reward with no length penalty (λ=0), the optimal policy never stops: there is always positive expected gain from emitting one more step. The fix implied is immediate and explicit: add a length penalty −λT to the reward.

Main Findings

Limitations and Open Questions

Several caveats deserve attention. The study is restricted to mathematical reasoning benchmarks; whether the same redundancy levels hold for coding, scientific reasoning, or open-ended generation is an open question. The sample sizes, while large for this kind of study, are still modest (60 GSM8K problems, 150 MATH-500 problems per model), and the authors are transparent about this. The external judge gap is large for some models, particularly Qwen3, which raises questions about what the external-judge measure is actually capturing versus the self-decoding measure.

The theoretical result is clean but stylised. The Advance/Idle/Stop model abstracts away the sequential structure of actual reasoning in ways that may matter for understanding the shape of the redundancy-difficulty curve, which the theorem does not predict. The authors flag this explicitly as an open problem.

The practical implication, adding an explicit length penalty to the reward, is well-motivated but the paper does not demonstrate that this actually works in practice or characterise how λ should scale with problem difficulty. The difficulty-aware token budget idea is mentioned but not tested. These are natural next steps rather than flaws in the current work.

What makes this paper valuable is not just the empirical numbers but the combination of rigorous definition, large-scale measurement, and a proof that locates the cause at the level of the training objective rather than any particular model or dataset. That framing has direct consequences for how the field should think about length penalties, budget-forcing techniques, and the design of reasoning-model training signals going forward. The finding that over-thinking is structurally optimal under current rewards, rather than a correctable quirk, should inform how seriously these efficiency concerns are taken at training time rather than inference time.

LLM ReasoningChain of ThoughtModel EfficiencyReinforcement LearningFrontier Models

Related Articles

LLMs Are Overconfident, Especially on Hard TasksWater-Filling Token Allocation for Reliable LLM WorkflowsLCF: Faster LLM Agent Communication via Latent Cache Transfer