The dominant assumption in language model development is that interpretability costs capability. You train the most powerful model you can, then attempt to understand it after the fact using probes, sparse autoencoders, or attribution methods that were never part of the training contract. A new paper, Scaling Inherently Interpretable Language Models, challenges this assumption directly. The authors argue that interpretability should be a constraint of the training pipeline itself, not an afterthought, and they back this up with scaling experiments across three orders of magnitude of compute.
The result is Steerling-8B, a diffusion language model that attributes any output to input tokens, human-understandable concepts, and training data, all through interfaces that are trained into the model rather than bolted on afterwards. The core claim is striking: interpretability metrics improve with scale rather than degrading, and the performance cost relative to opaque peers trained on substantially more compute is modest.
Why Post-Hoc Interpretability Falls Short
The paper opens with a careful diagnosis of why standard post-hoc methods are structurally limited. Probes report that information is decodable from a representation, not that the model uses it causally. Gradient-based feature attributions measure local sensitivity around a particular input, which may not correspond to the effect of a human-meaningful intervention. Sparse autoencoders reconstruct activations but do not guarantee that the recovered features are causal units in the computation. Chain-of-thought reasoning may describe a plausible rationale without being mechanistically tied to the computation that produced the answer.
The authors identify a deeper issue they call explanatory multiplicity, drawing on the Rashomon literature: two models with identical predictive performance can produce contradicting attributions for the same input. Any downstream action derived from a post-hoc explanation, whether steering, editing, or counterfactual reasoning, inherits this instability. The proposed remedy is to anchor the explanatory interface at training time, so that the concept library, absence baseline, and additive decomposition are shared constraints across all models the training procedure can produce.
Key Contributions
- A formal training recipe for inherent interpretability. The paper formalises five conditions that an attribution must satisfy to be considered faithful, then modifies each stage of the standard training pipeline to enforce them. Data, architecture, objective, loss, and evaluation are all touched.
- Atlas, a large-scale concept annotation pipeline. No suitable concept library existed at pretraining scale. The authors build one from scratch: millions of documents, hundreds of millions of free-form tags, canonicalised into over 33,000 concepts via clustering and deduplication, then annotated across more than one trillion tokens. A Qwen3-Embedding-0.6B encoder with a dot-product content head handles the positive-unlabeled supervision regime using masked binary cross-entropy and a non-negative PU loss.
- Causal diffusion architecture with an additive concept bottleneck. Steerling-8B uses block-causal attention that preserves diffusion-style parallelism within blocks while allowing KV caching across blocks. Between the transformer backbone and the language modelling head sits an additive concept bottleneck, making the logit decomposition algebraically exact. The masking objective provides a trained absence baseline, so feature-removal attributions are in-distribution by construction. The concept module accounts for 4% of parameters at 8B scale and would fall below 1% at frontier scales under the same parameterisation.
- IsoFLOP scaling sweeps across four model families. Autoregressive, causal diffusion, autoregressive with concepts, and causal diffusion with concepts are compared across three orders of magnitude of compute. The concept module shifts compute-optimal scaling exponents by a small, fixed per-backbone offset rather than introducing a growing penalty.
- Steerling-8B as a concrete instantiation. Trained on 1.2 trillion tokens with 150 billion midtraining tokens on a code and math augmented mixture, Steerling-8B lands within approximately 10% of average benchmark performance relative to open peer models trained on 2 to 16 times more compute.
Methodology and Results
The choice of diffusion over autoregressive generation is motivated carefully. Autoregressive models predict one token at a time under a strictly causal context, which makes it difficult to define a natural absence baseline for feature attribution and difficult to attribute concepts that span multiple tokens. Masked diffusion models predict multiple tokens jointly at each denoising step and have an explicit trained representation of missing information, which is precisely the baseline that faithful input attribution requires.
The scaling results are the most consequential part of the paper. Across all four model families, interpretability metrics improve monotonically with compute. The model predicts concepts more accurately, separates known from unknown representations more cleanly, routes more prediction mass through the concept bottleneck rather than the residual, and aligns concept embeddings more closely with human-meaningful labels as scale increases. The validation loss of Steerling-8B is predicted within 0.11 nats from small-scale fits using the joint Chinchilla form, and three of four interpretability metrics fall within tight predicted bounds. This is a genuinely surprising result: the usual expectation is that more capable models become more opaque, not more transparent.
The closed-loop intervention workflow is worth highlighting. Because each concept is represented as a direction in the model's activation space that lies on the computational path to the output, amplifying or suppressing a concept is a direct edit rather than an approximation. The paper describes a diagnosis-retrieval-correction loop: attribute an output to its concepts, retrieve similar training data via the concept geometry, and correct behaviour through concept steering without retraining. This is qualitatively different from post-hoc steering methods, which intervene on representations that were never trained to support such interventions cleanly.
Limitations and Open Questions
Several limitations deserve attention. The concept library, despite its scale, is derived from a specific corpus and annotation pipeline. Coverage of rare, highly technical, or culturally specific concepts is likely uneven, and the positive-unlabeled training regime means that absent labels are treated as unlabeled rather than verified negatives, which can inflate false positive rates in ways that standard metrics understate.
The 10% benchmark gap relative to opaque peers is presented as evidence that interpretability is not prohibitively costly, but the comparison is complicated by the fact that those peers were trained on substantially more compute. It is not straightforward to determine how much of the gap is attributable to the interpretability constraints versus the compute difference. A more controlled comparison at matched compute budgets would strengthen the claim.
The paper also does not address how the approach scales to instruction-tuned or RLHF-trained models, where the concept library and absence baseline may need to be adapted. The authors note that interpretability constraints are most effective during pretraining because finetuning restructures existing representations, but the practical implications for post-pretraining alignment pipelines are left open.
Finally, the human recoverability study, which evaluates whether humans can independently identify concepts from the model's characteristic evidence, is described but not fully detailed in the sections available. The reliability of this evaluation is important for the semantic faithfulness claims and warrants close scrutiny in the full paper.
Despite these caveats, this is a substantive contribution to a question that has been largely treated as settled. The claim that interpretability and capability are in fundamental tension is an assumption, not a theorem, and this paper provides the most systematic empirical challenge to it to date. Researchers working on mechanistic interpretability, model auditing, and AI safety will find the scaling results and the Atlas pipeline particularly relevant. The full paper is available at arxiv.org/abs/2608.07594.