One of the more persistent frustrations in training tool-use agents is the bootstrapping problem: you need good trajectories to train a good agent, but generating good trajectories requires a good agent. The standard workarounds, supervised fine-tuning on teacher-distilled data and sparse-reward reinforcement learning, each carry significant costs. The former locks you into whatever the teacher model can do; the latter struggles to provide meaningful learning signal across the long, multi-step interactions that real web search requires. DeepSearch-World (arXiv:2607.07820) proposes a third path: build a verifiable environment and let the agent teach itself.
Key Contributions
The paper makes two separable but tightly coupled contributions. The first is DeepSearch-World, a deterministic and reproducible environment for web-style search agents. The second is DeepSearch-Evolve, a self-distillation training framework that operates on top of it. The distinction matters because the environment is what makes the training framework tractable; without verifiability, self-distillation has no reliable signal to filter on.
- A 420K multi-hop question-answering task pool, constructed via entity-level random walks over knowledge graphs, designed to require genuine multi-step reasoning rather than single-retrieval shortcuts.
- Reproducible search and page-reading tools that return deterministic outputs, allowing trajectory replay and answer verification without relying on live web access.
- Explicit support for agentic cognitive behaviours: progress verification mid-trajectory, grounded reflection on retrieved content, and structured failure recovery when a search branch yields nothing useful.
- An iterative training loop covering trajectory generation, quality filtering, data mixing, and fine-tuning, with no distillation from a more capable external model at any stage.
Methodology
The self-distillation loop works roughly as follows. A base model generates search trajectories across the task pool. Because the environment is deterministic and the tasks have verifiable answers, trajectories can be automatically scored for correctness. High-quality trajectories are retained, mixed with existing training data, and used to fine-tune the same model. The process repeats, with each iteration producing a slightly stronger agent whose trajectories serve as the next round of training signal.
The critical enabler here is the environment design. Most web agent benchmarks rely on live or semi-live web access, which introduces non-determinism that makes trajectory replay and automated verification unreliable. By fixing the search index and page content, DeepSearch-World converts what is normally a noisy, partially observable process into something closer to a formal verification problem. This is not a new idea in isolation; verifiable environments have been central to RL successes in games and formal mathematics. Applying the principle systematically to open-domain web search at this scale is the contribution.
The multi-hop task construction via random walks is also worth attention. Random walks over entity graphs naturally produce questions that require chaining multiple retrieval steps, which is exactly the regime where single-step retrieval agents fail and where long-horizon reasoning matters most. The 420K scale gives the training pool enough diversity to avoid the agent overfitting to a narrow slice of query types.
The cognitive behaviour scaffolding, specifically progress verification, grounded reflection, and failure recovery, is built into the environment's action space rather than imposed purely through prompt engineering. This means the model can learn when and how to invoke these behaviours from trajectory-level supervision rather than relying on hand-crafted chain-of-thought templates.
Results
The 9B parameter model trained with this framework achieves the following on standard benchmarks:
- BrowseComp: 31.2%
- GAIA: 61.5%
- HotpotQA: 93.4%
These numbers are described as competitive with open-source agents of comparable scale. BrowseComp is the hardest of the three, requiring genuine multi-step web navigation rather than single-document retrieval, so 31.2% without any teacher distillation is a reasonable result. GAIA at 61.5% is the more telling figure given that benchmark's breadth across task types. HotpotQA at 93.4% is less surprising; the task pool's random-walk construction is conceptually similar to HotpotQA's design, so some degree of distribution alignment is expected.
The absence of teacher distillation is the headline claim, and the numbers broadly support it. The more interesting question, which the abstract does not fully address, is how the self-distilled model compares to a model trained with equivalent compute spent on teacher-distilled data from a larger model. That ablation would clarify whether self-distillation is genuinely competitive or merely a practical alternative when teacher access is unavailable.
Limitations and Implications
The deterministic environment is both the paper's strength and its most significant limitation. Real web search is non-deterministic: pages change, search rankings shift, and retrieval results vary across time. A model trained entirely in a fixed-index environment may develop brittle search strategies that assume stability. The gap between DeepSearch-World's controlled setting and live web deployment is not trivial, and the paper does not report results on live-web evaluations.
The random-walk task construction, while scalable, also produces a particular distribution of multi-hop questions that may not reflect the full diversity of real user queries. Questions generated by walking knowledge graphs tend to be well-formed and entity-centric; they may underrepresent the ambiguous, poorly specified, or domain-specific queries that make real-world search agents fail.
The broader implication is positive for the field. If verifiable environments can be constructed for other tool-use domains, the self-distillation approach generalises beyond web search. Code execution, database querying, and API interaction all have natural verification signals. The authors plan to release the environment, training pool, validation set, model, and code, which should allow the community to probe these questions directly.
For researchers working on agentic systems, the paper's core argument is worth taking seriously: the bottleneck to self-improvement is often not the training algorithm but the environment's capacity to provide reliable feedback. Investing in environment design may yield more reliable returns than searching for better RL objectives over noisy signals.
Full paper: arXiv:2607.07820