A persistent and underappreciated failure mode in large language models is their tendency to answer factual questions correctly in English while giving inconsistent or wrong answers to the same questions in other languages. This is not simply a vocabulary problem. The knowledge is often present in the model; it just cannot be reliably accessed through non-English interfaces. A new paper from researchers at University College London, Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning, takes a systematic look at this problem and proposes a training approach that meaningfully improves the situation without requiring expensive large-scale retraining.
The Problem and Why It Matters
Most capable open-weight LLMs are trained on corpora that are heavily English-dominated. They encode world knowledge primarily through English text, and while they can often translate fluently, the pathway from stored knowledge to correct non-English output is unreliable. Prior work has shown this inconsistency tends to emerge late in the forward pass: models sometimes retrieve the correct answer in intermediate layers but fail to map it into the target language correctly by the final output layers. This framing is important because it redefines the problem. Cross-lingual factual inconsistency is less about missing knowledge than about unreliable access to knowledge that is already there.
This distinction has practical implications for how you would fix it. If the knowledge is missing, you need more pretraining data. If the access pathway is broken, you might be able to repair it through targeted post-training. The paper tests this hypothesis directly.
Key Contributions
- PolyFact dataset: A fully parallel multilingual factual QA dataset grounded in Wikidata, covering 100K facts across 12 typologically diverse languages including high-resource languages like English, French, and Chinese, alongside lower-resource languages like Swahili and Bengali. The dataset is released openly.
- Systematic comparison of post-training methods: The paper evaluates light continual pretraining (CPT) on parallel text, supervised fine-tuning (SFT) on PolyFact, and reinforcement learning via Group Relative Policy Optimization (GRPO) on PolyFact, across two 7B-scale models: OLMo-2-1124-7B and Qwen-2.5-7B.
- Mechanistic analysis: Using interpretability tools (LAHIS and LAPE), the authors examine how each training method reshapes internal language routing, specifically looking at MLP layer specialization and attention head overlap across language pairs.
Methodology
The experimental design is clean and the choice of models is sensible. OLMo-2-1124-7B is an English-dominant base model with a large performance gap across languages, making it a good stress test. Qwen-2.5-7B has stronger multilingual pretraining, providing a comparison point where the baseline is already better.
All post-training uses LoRA (rank 64, alpha 128) to keep compute costs manageable. The CPT baseline trains on 235.5M tokens from the TED2025 parallel corpus. For GRPO, the reward function is designed to favour correct answers, penalise hallucinated outputs, and add a bonus when all 12 languages produce correct answers simultaneously. That consistency bonus is the key design choice: it explicitly incentivises the model to route the same factual knowledge through shared representations rather than learning per-language shortcuts.
The pure SFT baseline without a consistency term (lambda=0) is also evaluated, and the results are telling. Without the cross-lingual consistency signal, SFT models overfit to per-language performance and do not learn to share representations across languages. The KLAR unseen-language scores for pure SFT are notably weak, which supports the paper's central argument about the mechanism of improvement.
Results and Mechanistic Findings
The headline result is that GRPO consistently outperforms SFT across both models and across seen and unseen languages. CPT on parallel data provides limited additional gains when used alone, and stacking CPT before GRPO does not substantially improve over GRPO from the base model. This is a useful negative result: parallel text during continual pretraining improves representation alignment but does not reliably improve the final knowledge access step.
The mechanistic analysis is where the paper becomes particularly interesting. GRPO-trained models show:
- Reduced language specialization in MLP layers, meaning individual neurons are less dedicated to specific languages.
- Greater overlap in language-important attention heads across language pairs, suggesting more shared cross-lingual computation.
- SFT models show much less reorganization of this kind, consistent with the interpretation that SFT produces surface-level memorization rather than structural changes to multilingual routing.
This mechanistic evidence strengthens the paper's core claim considerably. It is not just that GRPO gets better numbers; the internal representations change in a way that is consistent with the theoretical account of why it should work.
Limitations and Open Questions
The authors are appropriately candid about the boundaries of their findings. The experiments are limited to two 7B model families, and it is genuinely unclear whether the same dynamics hold at larger scales or in mixture-of-experts architectures where language routing may work differently. The gains on more demanding benchmarks like Global-MMLU are limited, which suggests the method improves factual access rather than multilingual reasoning more broadly. That is a real constraint on the scope of the contribution.
There is also a data quality concern worth taking seriously. PolyFact is derived from Wikidata, which has uneven coverage across languages, regions, and entity types. The authors apply relation filtering and quality checks, but the potential for inherited biases to be more consistently expressed across languages after training is a genuine risk, not just a boilerplate disclaimer.
A few open questions follow naturally from this work. First, does the consistency reward need to operate over all 12 languages simultaneously, or would a smaller subset of typologically diverse languages produce similar gains? Second, the interaction between CPT and GRPO is somewhat underexplored; it is possible that CPT on factual rather than parallel text would combine better with GRPO. Third, the generalization to unseen languages is encouraging but the evaluation set is still relatively small. Understanding which linguistic properties predict transfer would be useful for practitioners choosing which languages to include in training.
Overall this is a well-motivated paper with a clear experimental design and a mechanistic story that holds together. The PolyFact dataset alone is a useful contribution to the field, and the finding that consistency-driven RL can improve cross-lingual factual recall without large-scale retraining is practically significant. Full details and released resources are available at the arXiv paper page.