Most work on language model failure detection treats the problem as binary: the model either gets the answer right or it does not. A new paper from Stanford argues this framing is too coarse, and that reasoning failures actually emerge through structurally distinct processes that leave measurable signatures in the chain-of-thought trace. The distinction matters practically: a model that commits early to a wrong answer requires a different detection strategy than one that remains genuinely uncertain throughout. Getting this wrong means applying self-consistency sampling when it cannot help, or skipping uncertainty signals when they would.
Key Contributions
The paper introduces a two-mode framework for characterising reasoning failures using token-level uncertainty signals computed over prefixes of chain-of-thought traces. The core insight is that failure is not a single phenomenon but two empirically separable ones:
- Committed failure: the model locks onto an incorrect reasoning path early. Uncertainty signals peak in predictive power at a identifiable commitment point T*, after which adding more of the trace actually degrades failure detection. Self-consistency applied here is actively misleading, since the model reproduces its wrong answer confidently across multiple samples.
- Persistent uncertainty: uncertainty accumulates throughout the trace without resolution. Predictive power increases monotonically with trace length, and the full trace is needed for best failure detection. Here, aggregating across completions is the appropriate strategy.
Critically, the framework requires only log probabilities from a single completion, not access to model weights or internal activations. This makes it applicable to closed-API frontier models such as GPT-4o and Gemini-2.5 Pro, which expose only token-level output distributions.
Methodology
The authors compute several token-level uncertainty signals (entropy, margin, NLL, nucleus probability, near-tie frequency) over prefix windows of fixed sizes: 128, 256, 400, 512, 1024, and 2048 tokens. For each window size T, a logistic regression classifier predicts failure from these aggregated features, evaluated using PR-AUC via out-of-fold predictions from stratified 5-fold cross-validation. PR-AUC is the right choice here given the variable and sometimes imbalanced failure rates across configurations (5% to 84%), where AUROC can be misleadingly optimistic.
The commitment point T* is defined as the prefix size at which PR-AUC is maximised before declining toward the full-trace baseline. The key falsifiable prediction is that committed configurations show a positive delta between early-peak PR-AUC and full-trace PR-AUC, while persistent configurations show monotonically increasing PR-AUC. Statistical evidence is pooled across configurations using four complementary tests: a sign test, Stouffer's Z, inverse-variance weighted meta-analysis, and a joint bidirectional sign test.
One methodological detail worth highlighting is the pre-final analysis. Failing models tend to produce longer reasoning traces, which could inflate full-trace uncertainty features simply by proxying trace length rather than capturing genuine uncertainty. The authors strip tokens occurring after the final answer marker before computing features, controlling for this length confound. This is a sensible precaution, though it implicitly assumes the answer marker is reliably identifiable across model outputs.
Evaluation spans 23 model-dataset configurations: nine models including Qwen3.5 variants (2B to 122B), Llama 3.1-8B, GPT-OSS-20B, Gemma4-31B, GPT-4o, and Gemini-2.5 Pro, tested across GSM8K, MATH-500, GPQA Diamond, and LiveCodeBench. The AR-LSAT benchmark was excluded because every configuration fell outside the applicable failure-rate band, a transparency-positive decision that also raises questions about how broadly the framework generalises to logical reasoning tasks.
Results
The framework's bidirectional predictions hold in 20 of 23 configurations. Fourteen show committed failure (sign test: 14/14, p = 6.1e-5), nine show persistent uncertainty. The pooled effect size for committed configurations is small but consistent: delta PR-AUC = +0.013, 95% CI [+0.005, +0.020]. These are not large effects, but their reproducibility across architectures, sizes, and domains is the more important claim.
The self-consistency analysis is practically valuable. Across three models on GPQA Diamond, the authors show that in committed regimes, single-completion uncertainty features can substitute for self-consistency on the most-confident inputs (top 30%) without sacrificing failure recall. Across both regimes, combining uncertainty features with self-consistency agreement rate yields consistent positive lift over either signal alone. This is a concrete, actionable finding for deployment settings where sampling 15 completions per query is expensive.
Limitations and Open Questions
The authors are commendably direct about the boundaries of their framework. Several deserve attention:
- The framework requires failure rates between roughly 15% and 60% with AUROC above 0.55. Outside this band, PR-AUC estimates become unreliable. This is a meaningful constraint: well-calibrated frontier models on easy benchmarks, or small models on hard ones, will often fall outside it.
- The commitment point is identified at six fixed window granularities, not at the token level. The reported T* is a window range, not a precise position. This matters if the commitment point is being used to make real-time decisions during generation.
- GPT-4o and Gemini-2.5 Pro expose only their top-20 log probabilities. Mean-based uncertainty features that depend on tail probability mass are unreliable under this truncation; the authors note that max-based features remain valid, but this limits the feature set for frontier models.
- The self-consistency evaluation covers only three configurations on one benchmark. The complementarity result is suggestive, not conclusive.
- The framework does not address whether chain-of-thought traces faithfully reflect internal computation. This is an orthogonal question, as the authors note, but it is a relevant one: if committed failure in the visible trace is epiphenomenal to the actual failure mechanism, the detection value of the signal is empirical rather than mechanistic.
The exclusion of AR-LSAT is worth a closer look. The stated reason is that every configuration fell outside the applicability band, which is a principled exclusion criterion. But logical reasoning tasks are precisely where understanding failure modes seems most important, and the consistent inapplicability suggests either that the framework's band assumptions do not hold for this domain or that the models tested are uniformly too weak or too strong relative to the task difficulty.
The broader research context matters here. Work using internal activations (probing, representation intervention) has shown that incorrect answers can be decoded before they are expressed in the output, and that early commitment restricts the effectiveness of later corrections. This paper's contribution is to show that structurally similar signatures are recoverable from the visible token distribution alone, without weight access. The two lines of work are complementary: mechanistic approaches explain why the signatures exist; this framework shows they are externally detectable at deployment time.
For practitioners building reliability layers around LLM pipelines, the key takeaway is operational: do not apply self-consistency uniformly. Characterise the failure regime first, then route accordingly. The framework provides a lightweight mechanism for doing this from a single completion, which is a meaningful reduction in inference cost relative to multi-sample approaches applied indiscriminately.
Full paper: arXiv:2606.06635. Code is available at github.com/sisl/LMTwoFailureModeFramework.