The problem of making large language models genuinely forget targeted knowledge is harder than it looks. Most existing unlearning methods produce what might be called cosmetic forgetting: the model stops producing harmful outputs under normal conditions, but a short fine-tuning run or a handful of in-context examples is enough to restore the suppressed behaviour. This matters enormously for open-weight models, where anyone with compute access can attempt recovery. RepSelect offers a principled diagnosis of why current methods fail and a technically grounded fix that consistently outperforms five established baselines across four model families.
Why Current Methods Fail
The paper's central insight is about representation overlap. When you perform gradient ascent on a forget set, the directions in weight space that carry the strongest signal for the targeted knowledge are, almost by construction, also high-variance directions for benign text. This happens because dangerous knowledge (biohazard protocols, abusive language patterns) shares substantial linguistic structure with ordinary language. Suppressing those high-variance directions therefore degrades general capability, and since those same directions are precisely what a fine-tuning attacker would recover when given a small relearn set, the forgetting is trivially reversible.
The authors formalise this as an "impossible triangle": forgetting, capability preservation, and robustness to relearning are simultaneously difficult to achieve because they pull in conflicting directions in representation space. Prior methods like GradDiff, NPO, and RMU all operate primarily in the high-variance subspace, which is why they trade off poorly between these three objectives. The empirical evidence is striking: baselines and fine-tuning attackers concentrate 50-62% of their weight-update norm in the top-50 principal components of the forget gradient, while RepSelect places only around 4% there.
The RepSelect Method
RepSelect (Representation Selectivity) is conceptually clean. Before each unlearning gradient update, it applies singular value decomposition to the forget-set weight gradients and collapses the top principal components, restricting weight changes to the low-variance, forget-specific subspace. The key claim is that while individual low-variance directions carry weaker forgetting signal than the top components, collectively they hold most of the forget-specific information while being less entangled with general capability and substantially harder for an attacker to recover.
Several properties make this practically attractive:
- No retain set is required, unlike GradDiff or RMU, which depend on retain-set regularisation
- A single pass over the forget set is sufficient
- The method applies to MLP modules across both dense and Mixture-of-Experts architectures without architectural modification
- It is data-efficient: 10 forget samples already achieve more than half the maximum unlearning gain on the tendency-unlearning task, with saturation around 90 samples
The disruption budget is controlled by stopping training when KL divergence on WikiText exceeds 0.01, roughly a 0.4% increase in negative log-likelihood. This ensures all methods are compared at matched utility, which is a methodologically sound choice that prevents the comparison from being confounded by different levels of capability degradation.
Results Across Models and Benchmarks
The evaluation covers two forget categories (biohazardous knowledge via WMDP-Bio, and abusive tendencies via BeaverTails animal abuse) and four model families: Llama 3.1 8B, Qwen 3.5 9B, Gemma 4 E4B, and DeepSeek-V2-Lite. The inclusion of MoE architectures alongside dense models is a genuine strength of the evaluation design, since prior unlearning work has largely focused on dense transformers.
The headline results are:
- Post-relearning answer probability reduction is 8-50x larger than the best baseline (NPO) on WMDP-Bio knowledge unlearning
- On Animal Abuse tendency unlearning, the reduction is 4-15x larger than NPO or RMU
- RepSelect is near-perfectly robust to few-shot attacks with k=5 and k=10 in-context examples
- MMLU accuracy and WikiText KL are maintained at levels comparable to baselines, confirming that the capability preservation claim holds
The authors note a compounding effect worth keeping in mind: a 10% per-token probability reduction over a 50-token answer translates to a greater than 99% reduction in full-answer probability. This means that even modest per-token improvements in unlearning quality have substantial practical consequences for usability of recovered knowledge.
Limitations and Open Questions
The limitations section is honest and points to several directions that remain genuinely open. RepSelect currently operates only on MLP modules. Whether the same SVD-based collapse applied to attention key/value projections would improve results or introduce new trade-offs is untested. Conceptual knowledge is plausibly encoded in MLP weights, but attention heads carry relational and positional information that may also encode domain-specific patterns.
The adversarial evaluation, while thorough relative to the literature, covers full fine-tuning and few-shot prompting. Other attack surfaces, such as activation steering, representation engineering, or model merging with a non-unlearned checkpoint, are left to future work. For open-weight models in a genuine adversarial deployment context, these are non-trivial gaps.
The forget sets used here are relatively small (567 samples for WMDP-Bio, 371 for BeaverTails). Scaling to larger forget sets, which would be necessary for anything approaching comprehensive biosafety or broad harmful-tendency removal, is acknowledged as future work. It is not obvious that the low-variance subspace remains cleanly separable from general capability representations at larger forget-set scales, where the forget distribution may overlap more substantially with benign text.
There is also a question about what "deep forgetting" means in practice. The paper measures robustness against fine-tuning on a held-out relearn set of similar size to the forget set. A well-resourced adversary with a larger relearn corpus, or one who combines fine-tuning with activation patching, may find different results. The framing of robustness as resistance to a specific attacker model is reasonable but worth keeping in mind when interpreting the claims.
Despite these caveats, RepSelect represents a meaningful step forward. The core observation, that the directions most effective for forgetting are precisely those shared with retain-set activations and most recoverable by an attacker, is both theoretically motivated and empirically well-supported. The method is simple enough to be reproduced and extended, with code and a Docker image provided. For researchers working on LLM safety, unlearning, or the broader question of how model weights encode knowledge, this paper is worth reading carefully.
Read the full paper on arXiv. Code is available at github.com/filyp/RepSelect.