When you call a reasoning model API, you are not simply buying access to a named model. You are purchasing a specific configuration: the model version, the reasoning effort setting, the output constraints, the price schedule, and the date on which that contract was in force. This paper, available at arXiv:2608.16956, makes that observation precise and tests one term of the contract experimentally: what happens, in cost and accuracy, when a buyer explicitly sets high reasoning effort versus simply omitting the parameter entirely on the same model?
The question is narrower than it might first appear, and that narrowness is a strength. Prior work, particularly Chen et al. (2026) on price reversal phenomena, established that listed token prices frequently misorder realized inference costs and that repeated calls to fixed queries vary substantially. This paper takes those findings as given and asks a different question: within a single model, does changing one request parameter produce a measurably different purchased outcome?
What the Study Does
The core design is a preregistered paired contrast. Claude Sonnet 5 was called with explicit high effort and, separately, with the effort parameter omitted entirely. Crucially, Sonnet 5's documented defaults are high effort with adaptive thinking, so these two cells are not expected to differ in their documented behavior. The experiment tests whether the explicit and defaulted forms of the same nominal setting produce the same delivered contract in practice.
Thirty AIME 2026 problems served as the task, each receiving five calls per cell. The full grid also included single-pass reference cells for GPT-5.6 Terra (effort omitted) and Claude Fable 5 (effort omitted), plus a small non-contemporaneous GPT-5.4-mini bridge collected earlier. Every call was assigned a terminal outcome under a frozen precedence taxonomy: correct, wrong, rail-exhausted with no valid answer, other no-answer, or provider failure. This matters because wrong answers and no-answers still cost money; treating them as missing observations distorts the true cost of a deployed contract.
The statistical plan, parser, spending ceiling, and outcome taxonomy were all frozen before outcomes were examined. Inference used item-clustered resampling, retaining all five calls per selected item, so repeated calls reveal within-item conditional behavior without being counted as additional independent observations.
Main Findings
- Mean delivered cost was $0.01031 higher per call under the explicit-high contract than under the omitted contract (95% interval: +$0.00204 to +$0.01974). The cost difference is real and detected.
- The accuracy contrast was +0.0133, but the study did not detect an accuracy difference. The registered interval runs from -0.0267 to +0.0467, meaning a gain of up to 4.67 percentage points cannot be ruled out.
- Cost per correct answer was $0.08665 under explicit-high and $0.07662 under omitted, as registered point estimates without separate intervals.
- Across all four main cells, descriptive cost-per-correct ordering placed Terra omitted lowest, then Sonnet omitted, Sonnet high, and Fable omitted. No dominance claim follows because cost and accuracy intervals overlap.
- The main grid produced no rail-exhausted no-answer outcomes, so the preregistered conservative gate fired and rail claims are bounded to preliminary existence cases and exact zero counts from the registered run.
Two item-level observations add texture without generalising. In one case, explicit high effort paired cheap wrong answers with a single costly correct computation, while the omitted contract produced five identical cheap wrong answers. In a preliminary item, one cell paired cheap wrong answers with full-rail censored no-answers. These are specific, not representative: they establish that qualitatively different terminal outcome mixtures can arise under fixed contracts on particular problems.
The Contract Census and Omission Semantics
A secondary contribution is a dated census of how different models handle omission and thinking control. The paper finds that omission semantics differ across models and, critically, within a single provider's model family. This means provider-level descriptions of default behavior are too coarse for buyers who need to reason about cost. A buyer cannot assume that omitting the effort parameter on two different models from the same provider produces the same realized contract.
The evidence grading here is asymmetric and honest. Where raw response structure confirmed that thinking tokens were generated, the claim is verified. Where responses were structurally indeterminate (such as a correct answer on a simple problem that could reflect either disabled thinking or adaptive thinking that selected zero tokens), the claim stays at documentation grade. The paper does not overclaim verification where the signal is absent.
Limitations and What This Does Not Settle
The study is explicit about its scope, and the limitations section is worth reading carefully. Several deserve emphasis here.
Thirty items from one competition-mathematics task family is not a probability sample from any well-defined population of model uses. The results apply to this cohort under the assumption of exchangeability with comparable contest problems, and should not be extended to other task types without new evidence. High observed accuracy across all cells also left limited headroom for detecting accuracy gains, though the registered interval does permit meaningful improvements.
The Sonnet cells were dispatched in a fixed high-before-omitted order rather than randomised. The design therefore cannot separate request-form effects from dispatch-position or short-run service variation. The authors are appropriately careful: they describe the contrast as descriptive of a specific session, not a general causal effect of setting high effort.
Latency summaries cover only the first 20 sequential calls per cell, not the full grid. The GPT-5.4-mini bridge is five items collected nine days earlier and supports no equivalence or population inference. And because billed hidden computation is opaque by design, the study can relate delivered cost to the contract and terminal outcome but cannot attribute the cost difference to particular reasoning content.
The broader implication the paper points toward is worth taking seriously. Effort choice and model choice are both dimensions of a contract selection problem, and current cost-aware routing frameworks (in the FrugalGPT lineage) treat model-level prices as known inputs rather than treating the configured contract as the unit of selection. A wider measurement program crossing an effort ladder within a model against a contemporaneous model ladder from the same provider could feed effort-aware cascade policies. That work remains to be done, and it would need remeasurement whenever the model menu or omission semantics change, which they do.
For practitioners, the immediate takeaway is concrete: on Sonnet 5 in July 2026, explicitly requesting high effort cost more per call than omitting the parameter, with no detected accuracy benefit and a cost-per-correct estimate that was also higher under the explicit contract. Whether that premium is worth paying depends on the buyer's valuation of the upper end of the undetected accuracy interval. That is a decision the study correctly leaves open rather than resolving with a recommendation it cannot support.