As LLM-based agent systems mature, a practical engineering reality has emerged: skills accumulate. What begins as a small library of reusable tool calls or task handlers grows, over time, into collections of hundreds or thousands of entries. The assumption baked into most current systems is that more skills means more capability. This paper, arXiv:2605.16508, challenges that assumption directly, asking whether there are principled laws governing how routing and execution performance degrade as skill libraries scale, and whether those laws can be used to guide optimisation.
This is a timely question. The broader scaling laws literature has focused almost entirely on model parameters and training compute. The behaviour of the system around the model, specifically the structure of the tool or skill interface, has received comparatively little formal treatment. This paper is one of the first serious attempts to fill that gap empirically.
Key Contributions
The paper identifies two coupled empirical laws across 15 frontier LLMs, 1,141 real-world skills, and over three million routing or execution decisions:
- Routing law: Single-step routing accuracy decays logarithmically with library size, with R² exceeding 0.97 across all tested models. This is a strong fit, and the consistency across model families is striking.
- Execution law: Before state realisation, joint routing accuracy is approximately multiplicative across steps. However, correct execution of a preceding step can improve difficult downstream routing decisions by roughly 4x, a substantial rescue effect.
- Coupling parameter: A single scalar, the logarithmic decay slope b, links both laws. Fitting routing-side behaviour predicts execution-side recoverability across models, suggesting the skill library itself, not just the model, is a first-class determinant of agent performance.
The practical payoff is real. Law-guided optimisation raises held-out routing accuracy from 71.3% to 91.7%, reduces what the authors call "hijack" by overly general skills from 22.4% to 4.1%, and transfers to downstream benchmarks, improving mean pass rate from 49.3% to 61.6% on ClawBench and from 28.4% to 34.5% on ClawMark.
Methodology
The experimental setup is impressively scaled for a systems-level study. The authors evaluate across 15 frontier models, which gives the routing law fits genuine generality rather than being artefacts of a single model family. The 1,141 skills are drawn from real-world sources, avoiding the toy-library problem that plagues many agent benchmarks.
The routing error taxonomy is one of the more analytically useful contributions. The authors distinguish three failure modes that emerge progressively as library size grows: local skill competition (semantically similar skills confusing the router), cross-family drift (the router pulling toward superficially related but functionally distinct skill families), and capture by "black-hole skills", overly general entries that attract routing decisions disproportionately. This progression from fine-grained to coarse-grained failure is a useful conceptual frame for practitioners debugging agent systems.
The multiplicative execution law deserves careful reading. The claim is that, absent state realisation, joint routing accuracy across a multi-step plan degrades multiplicatively, which is the expected result if steps are treated as independent Bernoulli trials. What is less obvious, and more interesting, is the asymmetric rescue effect: a correctly executed step can substantially recover downstream routing quality, implying that state information propagated through execution is genuinely informative for subsequent routing, not merely confirmatory.
Results and What They Mean
The logarithmic decay finding has direct implications for how agent systems should be designed. If routing accuracy follows log(N) decay, then doubling the skill library does not halve accuracy, but the degradation is real, predictable, and cumulative. For systems operating at hundreds of skills, this is already a meaningful performance tax. At thousands of skills, it becomes a primary bottleneck.
The black-hole skill phenomenon is worth highlighting separately. The authors find that certain skills, typically those with broad or underspecified descriptions, act as attractors in the routing space, capturing decisions that should go elsewhere. This is analogous to the mode collapse problem in generative models, but manifesting in retrieval. The reduction of hijack from 22.4% to 4.1% through law-guided optimisation suggests that skill description granularity and library curation policy are tractable levers, not just engineering hygiene.
The transfer to ClawBench and ClawMark execution settings is the most practically significant result. Routing improvements that do not transfer to downstream task completion are of limited value. The observed gains, while modest in absolute terms on ClawMark, are directionally consistent and suggest the routing law is not merely a measurement artefact.
Limitations and Open Questions
Several caveats are worth raising. First, the paper is a technical report rather than a peer-reviewed publication, so the methodology has not yet been subject to external scrutiny. The R² values for the routing law are impressive, but the details of how library size was varied, whether skills were added randomly or by some structured process, matter considerably for interpreting the fit quality.
Second, the coupling parameter b is shown to predict execution-side rescue across models, but the mechanism is not fully explained. It is not obvious why a scalar derived from routing decay should encode recoverability. A more formal account of why these two laws share a parameter would strengthen the theoretical contribution considerably.
Third, the study focuses on single-step routing accuracy as the primary metric. Real agent deployments involve dynamic skill addition, user-driven library growth, and context-dependent routing policies that may not behave as static library evaluations suggest. The laws identified here may hold in expectation but could break down under distribution shift in the skill library itself.
Finally, the ClawBench and ClawMark benchmarks are not widely established in the community. Replication on more standard agent evaluation frameworks would help situate these results relative to existing work.
Despite these caveats, the paper makes a genuine contribution. Treating the skill library as a structured object with measurable scaling properties, rather than a flat list of tools, is the right framing. The empirical laws are well-fitted and consistent across models. The practical optimisations derived from them work. For anyone building or studying LLM agent systems at scale, this is worth reading carefully.
Full paper: arXiv:2605.16508