Retrieval-augmented generation pipelines have a quiet failure mode that standard metrics do not capture. A retriever can return a document that is topically relevant but drawn from the wrong domain: a biomedical trial report surfaced for a finance query, or a nutrition abstract returned for a scientific-claim task. The fraction of top-ranked results from the wrong domain is what this paper calls the contamination rate, and in high-stakes settings where retrieved documents carry operational or legal authority, it is a first-order risk rather than a cosmetic nuisance.
The paper, available on arXiv, introduces C3R (Certified Cross-domain Contamination Control for Retrieval), a drop-in control layer that certifies a per-domain contamination budget using only an inferred domain posterior, with no query-time label required.
The Problem with Marginal Conformal Control
Conformal risk control (Angelopoulos et al., 2024) is the natural statistical tool here: it can bound the expectation of any bounded loss with finite-sample, distribution-free guarantees. The catch is that it delivers a marginal bound, averaged over all queries and all domains simultaneously. On the paper's primary testbed, contamination ranges from 0.01 on the cleanest domain to 0.63 on the most contaminated. A single marginal budget tuned to the average leaves the worst domain badly under-covered; tuned to the worst domain, it over-filters everything else.
Group-conditional conformal methods could in principle address this, but they assume the group membership is observed at test time. Production queries arrive unlabeled. Routing-based retrievers can infer a domain but carry no statistical guarantee on contamination. C3R is designed to close exactly this gap: per-domain certification from a latent domain assignment.
Methodology: Two-Split Conformal Transfer
The core of C3R is a two-split scheme built on risk-controlling prediction sets. The key technical contribution is a transfer bound that crosses from the inferred domain (what the router predicts) to the true domain (what the query actually belongs to), with fully estimable slack. This matters because any per-domain guarantee derived from an imperfect router must account for the probability mass the router misassigns.
The calibration procedure splits available queries into two disjoint sets. The first split estimates router error bounds, specifically the misrouted-mass terms that determine how much slack the transfer bound must absorb. The second split sets the filtering threshold for each domain. The probe itself, a temperature-scaled logistic head on frozen embeddings, is trained on a separate disjoint pool entirely, so there is no leakage between probe training and threshold calibration.
At deployment, C3R operates as a soft demotion layer: results inferred to belong to the wrong domain are demoted in the ranking rather than hard-filtered. When the certified budget is infeasible given the router's estimated error, the method abstains rather than silently violating the guarantee. This is an important design choice. The failure mode is explicit and auditable rather than hidden.
Heterogeneous budgets are supported natively. In the main experiments, the most contaminated domain is held to a strict budget of 0.05 while others are set to 0.15, which is a genuinely asymmetric allocation that reflects how practitioners would actually want to deploy such a system.
Results and What They Show
The headline result is clean: across 1,000 resampled calibrations on the primary BEIR-MIX testbed, C3R incurs zero per-domain certificate violations. Marginal conformal risk control, by contrast, violates the most-contaminated domain in every single draw, and a second domain in 53% of draws. A simulation across 360,000 checks reproduces this pattern.
Beyond validity, the soft-demotion mechanism retains substantially more recall than calibrated hard-filtering cascades at equal certified contamination. At tight budgets on contaminated domains, C3R retains up to roughly six times more Recall@10 than the strongest calibrated cascade. On clean domains, all methods perform equivalently, which is the correct behaviour.
Several additional findings are worth highlighting:
- Swapping in a stronger reranker (Qwen3-Reranker-8B versus the MS-MARCO cross-encoder) improves ranking quality but leaves contamination essentially unchanged. This confirms that contamination is orthogonal to ranking quality and that a dedicated certified layer addresses something rerankers do not.
- Contamination tracks whether a pool contains a same-subject cluster, not merely topically adjacent domains. A pool of four StackExchange technical forums remains nearly clean despite adjacency; pools mixing biomedical and scientific corpora show pronounced asymmetry.
- Inter-domain centroid cosine similarity in the BGE-base embedding space correlates with contamination level across the four constructed pools, providing a geometric interpretation of why some mixes contaminate more than others.
- An LLM-judged downstream probe across three readers finds that wrong-authority grounding rises with retrieval contamination and falls under C3R control, with 17% of contaminating retrievals judged as wrong-authority versus 0% for within-domain retrievals.
Limitations and Open Questions
The authors are candid about the method's ceiling. The tightest certifiable budget is bounded below by the router's misrouted-mass term. On a heavily contaminated domain with a strict budget, C3R will abstain rather than violate, which means the tight, feasible certificates land on clean and moderately contaminated domains. What the method delivers on the hardest domains is a certified reduction, not always a tight bound. Tightening it requires a better domain router, which is outside the scope of this work.
The BEIR-MIX evaluation uses general-purpose corpora. Sector-Bench, built from public federal regulations, closes part of the gap to regulated settings, but the paper is explicit that certifying contamination on provenance is not the same as certifying appropriateness of regulatory authority. A human-annotated appropriateness layer with inter-annotator agreement remains future work.
TREC-COVID contributes only 50 test queries, making its per-domain certificate a small-sample stress case that frequently abstains. The subject-overlap trend, while suggestive, rests on four constructed pools and is presented as a corroborating observation rather than a law. The naive transfer scheme's failure mode is established in simulation rather than on BEIR-MIX directly, because the probe quality is sufficient that naive transfer does not visibly break on this benchmark.
For the field more broadly, C3R raises a question that is worth sitting with: if contamination is orthogonal to ranking quality and standard metrics are blind to it, how many deployed retrieval pipelines are silently surfacing wrong-domain evidence at rates that practitioners would find unacceptable if they could measure them? The paper's CSCR evaluation protocol and BEIR-MIX assembly, which will be released alongside the calibration code, give the community the tools to start answering that question systematically.