A new paper from arXiv (arXiv:2606.24890) asks a question that should concern anyone thinking about how AI systems acquire their values: can a small, organised group of volunteers meaningfully shape what language models say about a contested topic, simply by editing Wikipedia? The answer, according to this study, is yes, and the effect is statistically robust across multiple attribution methods and model scales.
This matters because it closes a loop that has been theoretically obvious but empirically unexamined. Wikipedia is a standard, upweighted component of virtually every major pretraining corpus, including The Pile, RedPajama, Dolma, and the datasets behind GPT-3 and LLaMA. Organised Wikipedia editing by advocacy groups, PR firms, and state actors is well-documented. What this paper does is connect those two facts with actual measurements, showing that legitimate, policy-compliant editing campaigns translate into detectable shifts in model behaviour on the edited topics.
What the Paper Does
The authors study the Pro-Animal Wikipedians (PAW), a volunteer group that adds sourced animal welfare content to Wikipedia articles about fast-food companies, animal sentience, and related topics. PAW made 125 edits across 115 pages. Using those edits as a natural experiment, the paper quantifies their downstream influence on language model behaviour through two complementary gradient-based attribution methods.
The methodological core relies on the Bergson library, which implements two distinct attribution approaches:
- TrackStar: Computes gradient similarity between training documents and model queries. Applied to Llama 3.1 8B over a corpus of 160 billion tokens, this measures which training documents are most statistically associated with a given model output.
- MAGIC (counterfactual influence estimation): Backpropagates through the full training process to estimate what would change if a specific document were removed. Applied to Llama 3.2 1B across five random training-order seeds, this gets closer to a causal claim about document influence.
The experimental design is sensible. The authors compare PAW-edited content's attribution scores on animal welfare queries against its scores on general queries about the same companies. This within-entity contrast is the key control: if PAW edits were simply correlated with high-traffic pages, you would expect elevated attribution on both query types. Finding elevated attribution only on animal welfare queries is evidence that the model has learned the specific association PAW's edits encode, not just that these are prominent pages.
Results
The findings are consistent and, in places, striking:
- TrackStar on Llama 3.1 8B: PAW-edited sections constituted 68% of the highest-attributed documents for animal welfare queries (p < 0.0001), but only 52% for general queries about the same companies (p = 0.53, not significant).
- MAGIC on Llama 3.2 1B, across five random seeds: in every single seed, all 10 of the top-10 most influential documents for animal welfare queries were PAW edits. For general queries, the same top-10 sat at chance levels (4 to 6 of 10).
- Mean PAW influence exceeded control influence on animal welfare queries with p < 0.0001 in every seed, with effect sizes 6 to 30 times larger than on general queries.
- Leave-subset-out validation produced Spearman rho = 1.00 across all 10 runs, indicating near-perfect internal consistency.
The fine-tuning ablation adds a cleaner causal layer. Models trained on PAW content reduced perplexity on animal welfare text from 12.4 to 8.4; models trained on control content reduced perplexity on control text from 16.1 to 11.4. Critically, there was no crossover benefit: PAW-trained models did not improve on control text and vice versa. This specificity is important because it rules out the interpretation that PAW edits simply added more text, improving general performance.
Limitations and Open Questions
The authors are candid about the paper's constraints, and it is worth taking them seriously.
The attribution experiments run on models of 1B and 8B parameters. The authors argue, with some arithmetic, that dilution to frontier scale is more modest than raw token ratios suggest, because Wikipedia is upweighted (typically 2 to 5 times) and seen across multiple training epochs. Their estimate is that effective dilution from their experimental setup to a frontier model trained on 15 trillion tokens is roughly 2 to 5 times, not 15 to 30 times. This is plausible but remains an extrapolation. Direct attribution experiments on GPT-4 class models are not computationally feasible with current tools, so the frontier-scale claim is inferential.
The paper also lacks topic-matched controls. The comparison is between PAW-edited content and general content about the same companies, not between PAW-edited content and other animal welfare content written by non-PAW editors. This means the paper cannot fully separate whether the signal comes from PAW's specific framing choices or from animal welfare content more broadly. That distinction matters if you want to understand the mechanism rather than just the existence of the effect.
Perhaps the most important limitation is the gap between influence on model prediction quality (what MAGIC measures) and influence on what a deployed chatbot actually says in conversation. These are related but not identical. Retrieval-augmented generation, RLHF fine-tuning, and system prompts all sit between pretraining data and user-facing outputs. The paper demonstrates that PAW edits shape the model's internal representations; it does not demonstrate that they change the answer a user gets from ChatGPT when asking about McDonald's animal welfare policies.
Implications
The paper's framing as a practical guide for advocacy organisations is deliberate, and it raises questions that go beyond the technical results. The authors position PAW's editing as legitimate and policy-compliant, which it appears to be. But the same methodology is available to any organised group with an agenda, and Wikipedia's existing literature on coordinated editing campaigns (corporate PR, state-backed operations, political movements) suggests that not all such groups will be operating in good faith or within policy.
What the paper establishes empirically is that the path from organised Wikipedia editing to measurable LLM behaviour change is shorter than most people assumed. The cost is low: 125 edits by volunteers. The effect is specific: elevated attribution on precisely the topics the edits address. And the mechanism is already baked into frontier models, because those models were trained on Wikipedia before this paper was written.
For researchers working on training data governance, this is a concrete case study in how the provenance and editorial history of training sources matters. The standard assumption in dataset documentation is that Wikipedia is a high-quality, relatively neutral source. This paper complicates that assumption by showing that Wikipedia's content is itself the product of organised editorial campaigns, and that those campaigns have downstream effects on model values that are measurable and topic-specific.
The methodological contribution, using TrackStar and MAGIC together as complementary attribution tools within a single study, is also worth noting. One measures association, the other estimates causation; their agreement across this study strengthens the overall case considerably. This paired approach is a useful template for future data attribution work.
Full paper: arXiv:2606.24890