One of the persistent criticisms of clinical AI systems is that they produce outputs without exposing the reasoning behind them. A model that says "this patient has AMD" without showing its working is of limited use to a clinician who needs to make a defensible decision. This paper, available at arXiv:2607.09664, takes a structured approach to this problem by mapping the components of an AI-assisted retinal diagnosis onto the Toulmin model of argumentation, a framework from philosophy of rhetoric that has been used in legal and scientific reasoning for decades but rarely applied to medical AI pipelines.
The Toulmin model breaks an argument into six components: claim, grounds, warrant, qualifier, rebuttal, and backing. The authors argue, convincingly, that this structure maps naturally onto the information available during image-based diagnosis. Rather than treating the ML output as a black-box verdict, each component is populated by a different module, and the assembled argument is presented to the human clinician for critical review.
Key Contributions
The paper makes several distinct contributions worth unpacking separately:
- Architectural decomposition: The diagnostic pipeline is explicitly structured around Toulmin components. YOLOE object detection models provide the grounds (detected biomarkers). MedGemma, a medically adapted 27B multimodal LLM, acts as the warrant agent, linking those biomarkers to a diagnostic claim using its embedded medical knowledge. The qualifier is constructed from quantitative performance metrics of the detection models, essentially a calibrated uncertainty signal. The rebuttal is generated by retrieving visually similar reference cases using MedSigLip embeddings.
- Dual biomarker extraction: Two YOLOE models trained on different OCT datasets (AROI and OCT5k) are run in parallel. Their complementary strengths are explicitly acknowledged and incorporated into the qualifier, rather than being hidden from the downstream reasoner.
- Honest qualifier construction: The qualifier is not a vague confidence score. It is a set of explicit observations about known model failure modes, for example, the AROI model's tendency to classify drusen as PED, and the OCT5k model's lower recall on hard drusen. These are passed as text to MedGemma, which can then reason about the reliability of its own inputs.
- Separation of concerns between argumentation and explanation: The discussion section makes a useful distinction between argumentation (defending a position against disagreement), justification (providing support for acceptance), and explanation (increasing understanding). The framework targets all three, which is rarer than it sounds.
Methodology
The system was evaluated on 200 OCT images from the OCTDL dataset, split evenly between AMD and DME cases. Two tasks were defined: T1 (binary AMD detection) and T2 (AMD staging into early, intermediate, and late). Five experimental configurations were tested, varying the presence and composition of the rebuttal and qualifier components. MedGemma was used without any additional fine-tuning, which is a deliberate choice: the system is designed to work with a general-purpose medical LLM augmented by structured external evidence, not a task-specific fine-tuned model.
Raw YOLOE detections are translated into natural language summaries using Claude Opus 4.7 before being passed to MedGemma. This is a pragmatic design choice that deserves some scrutiny; it introduces an additional model into the pipeline whose errors are not explicitly tracked, though the authors do not dwell on this.
Case retrieval for the rebuttal uses MedSigLip, a SigLIP variant pre-trained on medical imagery, to compute cosine similarity between the query image and a reference case set. The top-3 retrieved cases are included if similarity exceeds 0.93, a threshold that proved consequential in the ablation experiments.
Results and What They Show
The quantitative results are instructive, if somewhat modest in absolute terms. MedGemma alone (Exp0) achieved precision of 0.56 and recall of 0.39 on T1, confirming that the base model struggles with OCT interpretation without external grounding. Adding the structured argumentation components improved this substantially. Exp1, which included similar case retrieval above the 0.93 threshold but no qualifier, achieved precision 0.79 and recall 0.93, the best T1 balance overall.
For T2 staging, late-stage AMD was the most reliably classified category across all experiments, which is consistent with the clinical reality that late AMD has more visually distinctive features. Early AMD performed poorly throughout, and the authors correctly attribute this to the AROI model's conflation of drusen with PED. Since PED presence is a staging criterion, this systematic error propagates directly into staging accuracy.
A few patterns from the ablation are worth noting:
- Including all four qualifier observations (Exp4) alongside unrestricted case retrieval improved late AMD recall to 0.943 but at the cost of T1 precision dropping to 0.604.
- Removing similar cases entirely (Exp3) gave reasonable T1 performance (precision 0.76, recall 0.90), suggesting the rebuttal component is helpful but not strictly necessary when the qualifier is well-constructed.
- Exp2, which included the qualifier but applied no threshold to case retrieval, produced the worst T1 precision (0.54) with 82 false positives, indicating that low-quality retrievals actively harm performance.
Limitations and Broader Implications
The authors are transparent about the main limitations. The reference case set is small, which constrains the quality of the rebuttal component. There is no subjective evaluation by clinical experts, which is a significant gap for a system explicitly designed to support clinician decision-making. The evaluation dataset (OCTDL) is also relatively clean by clinical standards, and performance on noisier real-world data remains unknown.
There is also a question about the intermediate LLM step used to translate YOLOE detections into natural language. Introducing Claude as a summarisation layer adds latency and a potential point of failure that is not evaluated. If the summarisation introduces errors or omissions, MedGemma's reasoning is compromised in ways that are difficult to audit.
The broader contribution of this work is conceptual as much as empirical. The argument that the input to an LLM reasoner, not just its output, should be structured according to an argumentation framework is a useful reframing. Most XAI work focuses on explaining what a model decided; this paper focuses on structuring what the model is told before it decides. That is a meaningful distinction, and the Toulmin model provides a principled vocabulary for doing it.
The framework is presented as general, and there is no obvious reason it could not be applied to other imaging modalities or diagnostic tasks. Whether the specific component assignments (YOLOE for grounds, MedGemma for warrant, MedSigLip for rebuttal) are optimal, or whether the Toulmin structure is the right argumentation model for all clinical contexts, are open questions. But as a proof of concept for structured, interpretable AI-assisted diagnosis, this is a credible and thoughtful piece of work.
The full paper is available at https://arxiv.org/abs/2607.09664.