Decentralized finance presents a genuinely hard problem for automated supervision: credit exposures shift rapidly across interconnected protocols, shocks propagate faster than manual inspection allows, and the cost of a false alarm in a high-stakes regulatory context is not symmetric with the cost of a miss. DeXposure-Claw attacks this problem by asking a pointed question that most LLM-agent papers sidestep entirely: what happens when you measure the false-intervention rate explicitly, against a regulator-aligned ground truth, rather than treating agent output quality as a proxy for safety?
The answer, it turns out, is sobering. Even a carefully architected pipeline that routes LLM decisions through structured forecasted evidence still misfires on roughly 37 to 44 percent of its intervention-level recommendations, depending on the model. That figure is the paper's most important result, and it deserves to be read before the F1 improvements.
Key Contributions
The paper introduces three things that are worth separating clearly:
- DeXposure-FM, a graph time-series foundation model built on the GraphPFN backbone with a LiMiX-16M transformer encoder, trained jointly across three prediction heads (edge existence, edge weight, and node total-value-locked change). It forecasts future exposure networks at horizons of 1, 4, 8, and 12 weeks.
- DeXposure-Claw, a four-layer agentic pipeline that uses FM forecasts to build typed evidence bundles (alerts, attribution signals, stress-scenario losses, uncertainty estimates), passes only that structured evidence to an LLM for ticket drafting, and then gates intervention-level outputs through data-health and confidence checks before release.
- DeXposure-Bench, a six-axis evaluation harness that scores forecast quality, warning behaviour, uncertainty calibration, stress-scenario fidelity, ticket quality, and robustness. Critically, the ticket-quality axis uses absolute exposure loss as ground truth rather than fractional changes, which avoids the common problem of overweighting small protocols.
The evaluation harness is arguably the most transferable contribution. The field has lacked a principled way to measure false alarms in agentic financial supervision, and the explicit false-intervention rate (FIR) metric fills that gap in a way that maps directly onto regulatory priorities.
Methodology
The architecture is deliberately decomposed to keep the LLM as a constrained drafting component rather than the release authority. Layer 1 runs the FM forecast. Layer 2 converts forecasts into a typed evidence bundle using deterministic monitors and stress scenarios. Layer 3 calls the LLM once per weekly cycle to produce ranked supervisory tickets with targets, severities, and rationales. Layer 4 applies data-health and confidence gates before any intervention-level ticket is released.
The data-health score averages four checks (freshness, missingness, topology, discontinuity) and the confidence score combines data health, Monte Carlo dispersion across forecast samples, and forecast horizon. Intervention-level actions are only feasible when both exceed configurable thresholds. This is the paper's primary safety mechanism, and the ablations confirm it is load-bearing in a way that the choice of LLM is not.
The evaluation runs on five years of weekly real DeFi exposure data, with a frozen 2025 test split of 29 weeks. Three LLM decision models are tested (Claude Opus 4.7, Claude Sonnet 4.6, Gemini 2.5 Pro) and a three-model judge panel scores explanation quality. The whole pipeline runs on a single RTX 4090 at well under a dollar of API spend per weekly decision cycle, which is a meaningful practical constraint for the target deployment context.
Results
The headline numbers require careful reading because the improvements are real but modest in absolute terms, and the failure modes are persistent:
- The full FM-grounded pipeline with gating (m7, Claude Sonnet 4.6) achieves F1 of 0.0288 against a persistence-rules baseline of 0.0076. That is a statistically significant improvement (p < 0.001) but the absolute F1 values reflect the genuine difficulty of the task over a large protocol universe.
- The false-intervention rate for the recommended deployment configuration is 0.374, with a 95% confidence interval of [0.224, 0.529]. Stronger models do not help: Opus 4.7 has FIR 0.437, worse than Sonnet 4.6 at 0.374, a difference that is not statistically significant.
- Gemini 2.5 Pro achieves lower FIR (0.190) but only by intervening far less frequently, at much lower F1 (0.0139). This is a conservative abstention strategy, not better calibration.
- The scenario engine is the single most load-bearing component: removing it collapses ticket precision to zero on clean data. The confidence gate is the primary safety mechanism: removing it raises FIR to 0.429 from the gated baseline. The data-health gate becomes critical under data degradation, suppressing false alarms substantially under 80 to 98 percent feature or edge masking.
- The FM forecaster does not win on point accuracy (persistence is better on PageRank MAE and rank correlation) but it is the only predictor that supplies trend signals and calibrated uncertainty estimates, which are what the downstream evidence layer actually consumes.
The warning system achieves lead times of 4 to 5 weeks on historical crises including Terra/Luna, FTX, and SVB/USDC, with precision of 1.0 across all alert budgets for SVB/USDC. These are pre-test historical results rather than held-out test performance, but they provide useful calibration for the monitor's behaviour on known events.
Limitations and Implications
The paper is admirably honest about what the system does not solve. Over-intervention persists at roughly 37 to 44 percent regardless of model choice, which means the system is unsuitable for autonomous high-severity action. The authors frame it explicitly as an auditable recall-and-explanation option for human-in-the-loop supervision, not a replacement for conservative rule-based systems. That framing is appropriate and the empirical results support it.
Several open questions remain. The F1 values, while significantly better than the baseline, are low enough that the practical utility of the ticket ranking over a large protocol universe warrants further investigation in a live supervisory context. The judge-panel evaluation of explanation quality shows inconsistent results across judge models (significant under GPT-5.5, not significant under Claude Opus 4.8, absent under Gemini), which limits confidence in the qualitative ticket assessment. The evaluation is also confined to a 29-week frozen test split, and DeFi market structure changes quickly enough that temporal robustness beyond this window is an open question.
For the broader research community, the most transferable ideas are the FIR metric and the DeXposure-Bench harness. The practice of evaluating agentic financial systems against fractional exposure changes, rather than absolute loss, systematically distorts which protocols appear risky and which interventions appear correct. Any system that targets regulatory supervision should be measuring false alarms explicitly against the loss metric that regulators actually care about. The code and harness are available at github.com/EVIEHub/DeXposure-Claw, which makes the evaluation infrastructure reusable independently of the full pipeline.
The practical recommendation from the ablations is clear: deploy m7 with Claude Sonnet 4.6. It is approximately five times cheaper than Opus 4.7, achieves higher F1, and the safety properties come from the gates rather than the model, so the model choice is purely an efficiency and coverage decision. That is a useful design principle for anyone building agentic systems in high-stakes domains more broadly.