Large language models can produce chain-of-thought (CoT) reasoning that reads as logically coherent while being causally disconnected from the premises it purports to use. This is not a minor theoretical concern. If a model reaches a correct answer through spurious correlations rather than genuine logical dependency, it will fail silently on distribution shifts and provide false confidence in high-stakes settings. A new paper, Interventional Grounding Audits: Black-Box Premise-Dependency Tests for LLM Chain-of-Thought via Predicate Substitution, proposes a principled, black-box protocol for detecting exactly this failure mode at the level of individual reasoning steps.
The Core Problem: Right Answer, Wrong Reasoning
Existing methods for evaluating CoT quality are largely passive. Self-consistency checks whether the final answer is stable across multiple samples. Attention analysis probes internal weights. Neither approach can establish whether a specific intermediate reasoning step causally depends on a specific premise. The distinction matters enormously: a model can achieve 100% self-consistency on a problem while one or more of its intermediate steps are effectively decorative, invoking a premise textually without being logically sensitive to its content.
The paper formalises this as a dependency detection task. Given premises and a CoT with numbered steps, the goal is to determine, for each step-premise pair, whether the step genuinely depends on that premise. The ground truth comes from ProntoQA, a synthetic multi-hop deductive benchmark where proof trees are known exactly, making it possible to evaluate detection methods rigorously.
Methodology: Predicate Substitution as Causal Intervention
The proposed method is conceptually clean. Take a single premise, replace its target predicate with a novel invented symbol (e.g., substitute "tumpus" with "glumpus"), re-run the model, and check whether the normalised conclusion of each reasoning step changes. If step S_i changes under intervention on premise P_j, the step is classified as dependent on that premise. If it does not change, it is classified as insensitive.
The approach operates on a two-layer architecture: an observation layer (raw model text) and a concept layer (normalised predicates in canonical form). Interventions happen at the observation layer; comparisons happen at the concept layer. This separation is important because surface text can vary without the underlying proposition changing, and vice versa.
Two substitution strategies are evaluated:
- Consistent substitution: the novel predicate replaces the target uniformly across all occurrences in the problem. This is the primary evaluator.
- Local substitution: the replacement is confined to the specific premise being tested. This is more sensitive to transitive dependencies but introduces more false positives from propagation effects.
A cascade filtering step combines both strategies, using local substitution to recover transitive dependencies while suppressing propagation false positives. The entire pipeline is black-box: it requires only the ability to query the model, with no access to weights, activations, or logits.
Results and What They Show
Evaluated on 50 ProntoQA problems with GPT-4o, the consistent-substitution auditor achieves F1 = 0.806 on all proof-tree dependencies and F1 = 0.885 on predicate-determining dependencies specifically. The self-consistency baseline achieves F1 = 0.343, with non-overlapping 95% bootstrap confidence intervals confirming the gap is not noise. The precision difference is especially stark: 0.794 versus 0.226. Self-consistency cannot identify which premises a step depends on; it simply flags all premises as dependencies for any step that is consistent across samples, which is why its recall is relatively high (0.715) but its precision is poor.
Key findings from the analysis:
- Recall on predicate-determining dependencies is 100%, meaning the auditor misses none of the dependencies that actually determine a step's conclusion.
- All 33 false negatives under the full F1 metric involve entity-introduction premises (e.g., "Alex is a wumpus"), which are structural premises whose predicate does not determine the step's conclusion. Consistent substitution has a documented blind spot here, which local substitution partially addresses.
- 66% of correctly-solved problems contain at least one step insensitive to a direct proof-tree dependency, the "right answer, wrong reasoning" (RAWR) signal. Self-consistency assigns perfect agreement scores to 14 of these 33 cases.
- 30 cases were identified where a step textually invokes a premise yet is demonstrably insensitive to it under substitution, a direct measurement of reasoning misrepresentation.
- F1 is stable across chain lengths (3 to 5 hops), showing no monotone degradation with problem complexity.
A brief evaluation on Claude Sonnet 4.5 shows the audit transfers with perfect precision on alignable steps (F1 = 0.872), though the coverage advantage over a simple string-diff baseline narrows on that model's more regular output format, suggesting the audit's relative value is model-dependent.
Limitations and Open Questions
The authors are admirably transparent about scope. ProntoQA is synthetic and formal; its steps are parsable and its proof trees are exact. The normalisation component that maps raw text to canonical predicate form will not transfer directly to free-form reasoning on natural language benchmarks like FOLIO or ProofWriter. Extending the method to those settings requires adapted normalisation strategies that the paper does not yet provide.
Parse coverage is a practical concern. For the combined protocol, 10.5% of aligned certificates were unparseable and excluded from primary metrics. A lower-bound analysis treating all unparseable cases as negatives yields F1 = 0.703, still well above the baseline but a meaningful reduction. Majority voting (k=3) is suggested as a straightforward path to improving precision by reducing stochastic variation, which accounts for 92% of false positives in the analysis.
The dataset of 50 problems is modest. Bootstrap confidence intervals confirm statistical significance, but broader coverage across problem types and models would strengthen the generalisability claims. The paper is also honest that the 8% of metric-counted false positives that reflect transitive rather than direct dependencies are arguably correct under a looser dependency definition, pointing to a genuine ambiguity in how dependency should be formalised.
From a broader perspective, this work sits at the intersection of causal inference and LLM evaluation. The connection to Geiger et al.'s interchange interventions on internal representations is worth noting: both approaches use intervention to establish causal claims, but this paper's black-box framing makes it applicable without model access, which is practically significant given API-only deployment contexts. The RAWR finding, that a majority of correctly-solved problems contain at least one step not genuinely grounded in its stated premise, has direct implications for any application where reasoning transparency matters, not just the final answer. Reproducibility is handled well: all certificates, outputs, and scripts are publicly available, and every reported number can be recomputed without an API key.