Most foundation model post-training pipelines depend on something external: human preference labels, gold-standard answers, executable verifiers, or a stronger teacher model. This dependency is a genuine bottleneck. Labels are expensive, verifiers are task-specific, and for open-ended generation tasks like dialogue or summarisation there is often no reliable automatic oracle at all. A growing body of work since 2023 has tried to sidestep this by deriving update signals entirely from the model's own outputs. This survey, available at arXiv:2608.24982, is the first to systematically catalogue and classify these methods under a single framework.
What Counts as Unsupervised Post-Training?
The authors are careful about scope. They define Unsupervised Post-Training (UPT) with four boundary checks: the method must produce a real parameter update (ruling out pure inference-time tricks), the learning signal must come from same-lineage model artifacts (ruling out external verifiers, human labels, and stronger teachers), and the classification axis must be the internal object that produces the update signal rather than the downstream task type. This is tighter than most prior surveys on LLM self-improvement or test-time adaptation, which tend to mix inference-time and training-time methods, or allow external reward signals without flagging them as such.
The formal setup is clean. A sampling operator produces an internal update object z from the model and an unlabeled input. A signal extractor maps z to a scalar, pairwise, or token-level signal. An update operator then modifies the model state. The taxonomy classifies methods by the type of z, not by the loss function or the downstream task. That choice turns out to be analytically useful because the type of update object determines where error can enter the feedback loop.
The Four-Family Taxonomy
The 80 catalogued methods fall into four families:
- Prediction-Statistic Optimization: The update signal is a statistic of the model's predictive distribution, such as NLL, entropy, or confidence. Methods like continued pre-training with language modelling loss and entropy-regularised RL fall here. The feedback path is short but the signal is purely distributional and carries no semantic content about output quality.
- Sample-Relation Supervision: Multiple rollouts are compared and the relation between them (majority vote, semantic clustering, pairwise agreement) generates the signal. TTRL and EMPO are representative. This family is currently the most active in the literature, partly because majority voting over self-generated outputs provides a surprisingly strong proxy for correctness on structured tasks.
- Self-Generated Target Bootstrapping: The model produces pseudo-labels, rationales, or a curriculum that then serves as supervised targets. Self-Improve and R-Zero sit here. The feedback path is longest and the risk of compounding errors is highest, because the model is effectively training on its own mistakes if the initial generations are poor.
- Internal Evaluator Bootstrapping: A self-judge or internally trained reward model scores outputs, and those scores drive the update. Self-Rewarding LM is the canonical example. The evaluator and the policy share parameters or lineage, so reward hacking is a structural risk rather than an edge case.
An orthogonal two-axis view crosses Input Visibility (when during training the target inputs become visible) with Update Persistence (whether updates are discarded after each input or accumulated globally). This second view maps deployment regimes rather than signal types, and the combination of both axes gives a reasonably complete picture of the design space.
Error Propagation as a Unifying Theme
The most analytically valuable part of the paper is the treatment of error propagation. Each family has a characteristic feedback loop. In Sample-Relation Supervision, an imperfect consensus signal selects outputs; the update concentrates probability on those outputs; the next round's consensus is drawn from a more biased distribution. In Self-Generated Target Bootstrapping, the pseudo-label quality degrades as the model drifts from its initialisation. In Internal Evaluator Bootstrapping, the evaluator and the policy co-evolve, and there is no external check on whether the evaluator's scores remain calibrated.
The authors frame this as a common error chain across all four families, with the family determining where the loop begins and which safeguard can interrupt it. This framing is more useful than a simple accuracy comparison across methods, because it tells you what to monitor in deployment rather than just which method performed best on a benchmark. The practical implication is that UPT methods should be matched to tasks where the proxy signal is structurally reliable, and independent held-out evaluation is necessary regardless of which family is used.
Limitations and Open Questions
The inventory is frozen at May 2026, which the authors acknowledge. More substantively, the survey supports mechanism-level synthesis rather than pooled effect-size estimation: each method is assessed against its own backbone, budget, and metric, so cross-method comparisons are qualitative. That is probably the right choice given how heterogeneous the evaluation protocols are across the literature, but it means the survey cannot answer the question of which family works best in aggregate.
Several open questions are worth flagging. First, the boundary between Sample-Relation Supervision and Self-Generated Target Bootstrapping is sometimes thin; hybrid methods are assigned to their primary update object, but that assignment can be ambiguous for methods that use consensus to filter and then fine-tune on the filtered outputs. Second, the survey covers text-only and multimodal models and notes that the dominant mechanisms are modality-agnostic, but the evidence for this claim in the multimodal case is thinner than for text. Third, and most importantly, the paper does not evaluate the safeguards it recommends. The advice to use held-out evaluation and red-teaming is sensible but the effectiveness of specific safeguard designs against specific error chains remains an open empirical question.
For practitioners, the Input Visibility x Update Persistence framework is probably the most immediately actionable contribution. It provides a principled way to decide whether a given UPT method is appropriate for a given deployment setting, independent of the benchmark results reported in the source papers. For researchers, the error-chain analysis points toward a relatively underexplored direction: designing UPT methods where the proxy signal's failure modes are bounded by construction rather than mitigated post hoc.
The full paper is available at https://arxiv.org/abs/2608.24982.