Circuit tracing has emerged as one of the more promising approaches to mechanistic interpretability, offering a way to inspect the internal computation of language models through attribution graphs. The technique is compelling, but it has a practical bottleneck: after generating an attribution graph, a researcher must manually group individual features or MLP neurons into supernodes, sets of components that correspond to coherent semantic concepts. This step takes one to two researcher-hours per graph. A new paper, LLMs Can Annotate Attribution Graphs, addresses this directly with a simple, low-cost pipeline that automates supernode grouping using an LLM.
The timing is relevant. Recent work from Anthropic and others has produced increasingly detailed circuit analyses, but scaling that work requires reducing the annotation burden. This paper sits squarely in that gap, asking a practical question: how much of the manual annotation process can be automated without sacrificing interpretive quality?
Methodology
The pipeline is deliberately simple. For each feature in a pruned attribution graph, the authors first generate a natural language description from the feature's top-activating text examples, following standard autointerp practice. These descriptions are then passed to GPT-5 mini, which groups them into supernodes. A refinement step consolidates redundant groupings. The whole process costs between 3 and 61 cents per graph depending on pruning threshold, with denser graphs (around 585 nodes) at the upper end.
All experiments use Gemma-2-2B with Gemma Scope single-layer transcoders, and attribution graphs are generated via the circuit-tracer codebase. The authors vary the cumulative node-influence pruning threshold between 0.4 and 0.7 across experiments, with the tighter threshold retaining roughly 40 to 80 high-influence nodes per graph.
One thing worth noting about the design: the pipeline is intentionally naive. It operates purely on feature descriptions and ignores spatial information such as layer position and prompt-token location, the attribution flow between features, and the backward-from-output search style that experienced human annotators tend to use. This is acknowledged as a limitation, but it also makes the results more striking. Even without that structural context, the pipeline performs well.
Key Contributions and Results
- Interpretability parity with human annotators. On automated interpretability metrics (feature detection and text detection), supernodes generated by the pipeline match or slightly exceed human-annotated ones across 15 reference attribution graphs. Human-annotated supernodes score around 66.6% on feature detection; the full pipeline scores 93.4% with GPT-5 mini as judge.
- Strong performance on a structured task. On the two-hop Capitals task (e.g., "The capital of the state containing Dallas is"), the pipeline correctly recovers a supernode corresponding to the intermediate reasoning hop in 97 of 100 prompts. This is a meaningful result because the intermediate hop is precisely the kind of latent computation that circuit tracing is designed to surface.
- Scalability proof of concept. The authors automatically annotate 1000 attribution graphs from Wikipedia prompt completions and use an LLM judge to flag graphs with interesting or surprising structure for human review. This is a promising direction for unsupervised exploration of model behaviour at scale.
- Consistency across LLM judges. Results hold across GPT-5 mini, Gemini 2.5 Flash, and Claude Haiku 4.5, with GPT-5 mini and Gemini showing near-identical scores. Claude Haiku shows slightly lower absolute values but preserves the same relative ordering, suggesting judge bias is not a significant confounder.
Limitations and Open Questions
The paper is candid about where the approach falls short. The most significant limitation is scope: only a single model (Gemma-2-2B) and transcoder suite are studied, on a small set of tasks. The 2-hop Capitals task is relatively clean and well-structured; arithmetic queries in the appendix reveal that the pipeline produces supernode labels that are too coarse to capture known internal structure. This suggests the method works well when the underlying computation has clear semantic correlates, but may struggle with tasks where the relevant features are more entangled or numerically grounded.
The pipeline's naivety is both a strength (it's cheap and simple) and a weakness. Human circuit-tracers draw on graph topology, the original prompt, and the relationships between features. The current approach discards all of that. Incorporating attribution flow and positional information seems like a natural next step, and the authors flag this explicitly.
There is also a question of what "interpretable" means in this context. The automated metrics used here, feature detection and text detection, are reasonable proxies but are themselves LLM-based evaluations. The paper checks for judge consistency across models, which is reassuring, but the metrics remain indirect measures of genuine interpretability. Whether supernodes that score well on these metrics actually correspond to the computations a researcher would care about is harder to verify systematically.
The comparison between MLP-neuron and transcoder decompositions in the appendix is a useful first step toward a more principled evaluation of different feature bases, though it remains preliminary.
Implications for Mechanistic Interpretability
The broader significance here is not that this pipeline is the final word on automated circuit tracing, but that it demonstrates the annotation bottleneck is more tractable than it might appear. A naive LLM-based grouping approach, ignoring graph structure entirely, already matches human annotation quality on standard metrics. That is a useful calibration point for the field.
The open-ended exploration application is arguably the most interesting direction. If attribution graphs can be annotated cheaply at scale, it becomes feasible to run systematic searches for unusual or unexpected computational patterns across large prompt distributions. That kind of unsupervised discovery is difficult to do manually and could surface behaviours that targeted evaluations miss.
The work also connects to a broader question in interpretability research about how much semantic structure in neural networks can be recovered from surface-level descriptions alone. The answer here seems to be: quite a lot, at least for models and tasks where features are relatively clean. Code and data are available via the links in the paper for those who want to build on this directly.
Read the full paper at arxiv.org/abs/2608.02632.