A persistent assumption in the design of language model agents is that feedback improves performance. If a model gets something wrong, telling it what went wrong should help it do better. This seems obvious, but it has rarely been tested with the rigour the claim deserves. A new paper, "What Drives Interactive Improvement from Feedback?", takes that assumption apart carefully and finds that the picture is considerably more complicated than the standard narrative suggests.
The core problem is one of confounding. When a language model improves over multiple turns of interaction, that improvement could reflect genuinely useful corrective feedback, or it could reflect nothing more than the fact that the model got another attempt. More compute, more samples, format corrections on retry: all of these produce higher accuracy without any meaningful feedback processing. Disentangling these effects requires controlled experimental design that most prior work has not provided.
Methodology and Experimental Design
The authors introduce a student-teacher protocol across four verifiable reasoning environments: Omni-MATH (mathematical olympiad problems), Codeforces (competitive programming), BBEH Linguini (linguistic reasoning), and ARC-AGI1 (abstract visual reasoning). Verifiable tasks matter here because they allow objective measurement of whether a revised answer is actually correct, removing the need for subjective quality judgements.
Thirteen open-weight models spanning four families (Gemma4, Qwen3.5, Llama-3.x, and GPT-OSS) were evaluated in a dense cross-product matrix where every model acts as both student and teacher. This produces 169 student-teacher pairs per environment, giving sufficient statistical coverage to separate student effects from teacher effects. The protocol varies several dimensions independently: whether feedback is external or self-generated, how much interaction history is visible, task difficulty, and whether the teacher has privileged access to ground-truth information about the task.
Crucially, all conditions are compared against a repeated-attempt baseline, where the student simply retries without any feedback. This is the methodological move that most prior multi-turn evaluation work omits, and it turns out to matter enormously.
Key Findings
- Multi-turn improvement is not the same as feedback use. A substantial portion of accuracy gains across turns can be attributed to resampling rather than to the feedback itself. This means that reported multi-turn improvements in the literature may be significantly overstated as evidence of genuine feedback processing.
- Self-feedback adds little beyond self-refinement. When a model generates its own feedback and then revises based on it, performance is largely indistinguishable from simply retrying without feedback. The self-critique loop, popular in recent agent designs, does not appear to provide meaningful signal in these controlled conditions.
- High-quality external teachers do produce feedback-specific gains. The strongest teachers generate improvements that cannot be explained by resampling alone. But this effect is concentrated at the high end of teacher quality; mediocre external feedback performs similarly to self-feedback.
- Student identity explains more variance than teacher identity. The dense interaction matrices show a clear vertical separation by student in multi-turn trajectories. Which model receives the feedback matters more than which model generates it, though teacher choice is not irrelevant for a fixed student.
- Most gains arrive in the first feedback turn. Marginal improvements diminish sharply after the first corrective interaction. Later turns contribute progressively less, suggesting that long rollout budgets are mainly useful for difficult problems with slow recovery dynamics rather than as a general strategy.
- Longer histories and privileged teacher information are selective interventions. Providing the teacher with ground-truth information, or giving the student a longer interaction history, helps in some task-model combinations and not others. These are not universally beneficial design choices.
Why the Student-Side Bottleneck Matters
The finding that student capability dominates interactive performance has significant implications for how feedback-based systems should be designed and evaluated. Current practice often treats feedback generation as the primary engineering challenge: if we can produce better critiques, agents will improve more. This paper suggests the bottleneck is frequently on the other side. A student model that cannot translate a correct diagnosis into a better solution will not benefit from even an excellent teacher.
This reframes the problem. Rather than optimising feedback generation in isolation, researchers and practitioners should be asking whether their models are actually capable of acting on feedback at all. The authors introduce a normalised gain metric, ngain@10, which captures the fraction of initially unsolved problems that a model recovers through interaction. First-turn accuracy and feedback uptake turn out to be largely orthogonal capabilities: a model that performs well on initial attempts does not necessarily improve much through interaction, and vice versa.
This orthogonality is practically important. It means that standard single-turn benchmarks do not predict interactive performance, and that models selected for deployment in agentic settings based on single-turn evaluations may be poorly suited to the actual task they will face.
Limitations and Open Questions
The authors are candid about what their protocol does not cover. ARC-AGI1 receives less thorough ablation analysis than the other environments, and the authors acknowledge that more work is needed to rule out formatting artifacts and leakage effects in that setting. The study is also confined to zero-shot evaluation with short-horizon tasks and open-weight models, which limits how directly the conclusions transfer to fine-tuned systems, long-horizon agentic tasks, or closed-source models.
The zero-shot constraint is both a strength and a limitation. It makes the results clean and reproducible, but models trained specifically to use feedback (through reinforcement learning from feedback signals, for instance) might show different patterns. The authors identify this as the most natural direction for follow-on work: using the diagnostic protocol not just to measure feedback use but to train models that become better feedback users over time.
There is also a question about task coverage. The four environments chosen are all verifiable and relatively short-horizon. Computer use, code debugging across large codebases, and multi-step planning tasks all involve feedback integration over longer timescales and with partial observability. Whether the student-side bottleneck persists in those settings is an open empirical question.
For practitioners building multi-turn agents today, the immediate takeaway is straightforward: always evaluate against a repeated-attempt baseline before claiming that feedback is responsible for observed improvements. The framework released alongside this paper at j-lojek.github.io/feedback-generation-is-a-bottleneck provides the tooling to do this systematically. That alone makes this a useful contribution to a field that has been somewhat credulous about the benefits of feedback loops.