The prospect of AI systems that meaningfully improve themselves has occupied researchers since at least I. J. Good's 1966 speculation about an intelligence explosion. What has changed recently is that the prospect is no longer purely theoretical. Foundation-model-based agents are being deployed in software engineering, web navigation, and scientific discovery pipelines, and several of these systems incorporate mechanisms that update their own behaviour between interactions. The field has grown fast enough that terminology has fractured badly: the same underlying mechanism might be called self-correction in one paper, meta-prompting in another, and self-play in a third. This survey, available at arXiv:2607.13104, attempts to impose order on that fragmentation by offering a unified formal framework and a two-pathway taxonomy covering both parameter-level and structural forms of self-improvement.
What the Paper Contributes
The core contribution is a formal definition of a foundation-model-based agent as a tuple pairing neural parameters with an operational scaffold. The scaffold decomposes into four components: structured prompts, memory mechanisms, tool interfaces, and control logic such as routing or safety constraints. Self-improvement is then formalised as a self-induced update operator that modifies either the parameter set or the scaffold, or both, using signals generated by the agent's own execution rather than externally labelled data.
This framing is more precise than most prior treatments. It cleanly separates the target of modification (parameters versus scaffold components) from the source of the learning signal (intrinsic generative demonstrations, intrinsic evaluative feedback, or extrinsic exploratory experience). That separation matters because it makes mechanistic comparisons tractable across methods that previously appeared unrelated.
The two primary improvement pathways are:
- Foundation model improvement: gradient-based updates to model weights, driven by self-generated training signals. This pathway is slower and computationally expensive but produces stable, globally amortised capability gains. The survey subdivides this by signal type: synthetic demonstrations the model generates for itself, evaluative feedback from internal critics or verifiers, and reward signals collected through environmental interaction.
- Scaffold improvement: non-parametric modifications to prompts, memory, tool interfaces, or the full control architecture. This pathway is faster and more easily reversible, making it attractive for deployment contexts where rollback matters. Methods here range from automatic prompt optimisation to memory consolidation schemes to tool synthesis and full multi-agent architecture redesign.
The paper also provides an unusually thorough historical account, tracing the lineage from Schmidhuber's 1987 self-referential learning framework and the Gödel Machine through Neural Architecture Search and self-play reinforcement learning, and connecting these to modern in-context learning via the formal equivalence between the attention mechanism's key-value cache and the fast-weight programmers from the early 1990s. That historical grounding is genuinely useful rather than decorative; it clarifies which problems are genuinely new and which are classical problems in new clothes.
Methodology and Taxonomy
The survey's taxonomy organises roughly three years of rapid empirical work into a consistent reference frame. For foundation model improvement, the three signal subcategories (generative demonstrations, evaluative feedback, exploratory experience) map onto recognisable families of methods: self-play data generation and STaR-style bootstrapping fall under the first; RLHF variants, Constitutional AI, and consistency-based self-critique under the second; ReAct, Reflexion, and RL-from-environment-rewards under the third. The taxonomy is not presented as mutually exclusive, which is honest, since many deployed pipelines mix signal types within a single update cycle.
For scaffold improvement, the coverage is similarly structured. Prompt optimisation methods are separated from memory evolution schemes, which are in turn separated from tool governance (including tool synthesis and interface adaptation) and full-scaffold redesign approaches that modify multi-agent topologies or control flow. The survey is careful to distinguish evaluation environments such as SWE-bench from genuinely self-improving systems; a system that iterates within a single task instance without persisting changes to parameters or scaffold across interactions does not qualify under the formal definition, even if it looks superficially adaptive.
The comparison table against three related surveys (Gao et al. 2026, Fang et al. 2025, Tao et al. 2024) is useful for positioning. The main gaps in prior work are the absence of a unified formal treatment covering both pathways simultaneously, and the lack of historical grounding. This survey addresses both.
Open Problems and Evaluation Gaps
The survey's treatment of evaluation is one of its stronger sections. It distinguishes mechanism-level benchmarking, where specific components of the improvement pipeline are tested in isolation, from end-to-end evaluation of deployed self-improving agents across interaction horizons. The latter remains genuinely hard. Most existing benchmarks measure performance at a single point in time rather than tracking capability trajectories across improvement cycles, which means they cannot directly measure whether a system is self-improving in the sense the survey defines.
Several open problems stand out:
- Reward hacking and distributional drift are especially acute when the agent's own outputs form the training distribution. Methods that filter or reweight self-generated signals exist, but principled criteria for when to trust self-generated data remain underdeveloped.
- Reversibility is asymmetric across the two pathways. Scaffold changes can in principle be rolled back; parameter updates generally cannot without reverting to a prior checkpoint, which discards all subsequent learning. The survey flags this but does not resolve it.
- Safety considerations for self-modifying systems are discussed in the final section. The classical concerns about reward tampering and self-deception identified by Orseau and Ring remain relevant, and the survey argues they become more pressing as update cycles shorten and human oversight decreases.
- Cross-task generalisation from self-improvement is poorly understood. A system that improves on a specific task distribution may degrade on out-of-distribution tasks if the update operator overfits to the signals available during training.
Implications for the Field
The practical value of this survey is in giving researchers a shared vocabulary. The field has been producing results faster than it has been building conceptual infrastructure, and the cost of that is duplicated effort and missed connections between communities working on what are formally the same problem. The formal framework here is not overly complex; the agent tuple and the update operator notation are accessible to anyone working with language model systems. Whether the taxonomy will stick depends on how widely it gets adopted, but it is a reasonable proposal.
One limitation worth noting is that the survey covers work up to early 2026 and the field is moving quickly enough that some of the frontier results cited may already have been superseded. The authors address this by maintaining a companion GitHub page with technical updates, which is a sensible approach for a living survey in a fast-moving area.
For anyone working on autonomous agents, continual learning, or alignment of deployed systems, this is a useful reference. The historical sections alone are worth reading for the connections they draw between classical meta-learning theory and modern practice. The full paper is at https://arxiv.org/abs/2607.13104.