Most post-training work on large language models concentrates on deductive tasks: mathematics, code generation, formal proof, domains where a correct answer exists and can be verified automatically. The implicit assumption is that reasoning means deriving a single right answer through deterministic steps. But a large class of practically important reasoning problems does not fit this mould. Scientific inference, medical diagnosis, forecasting, and everyday decision-making all require drawing uncertain conclusions from sparse, noisy observations. This is inductive reasoning, and current LLMs handle it poorly.
A new paper, available on arXiv, proposes a method called Program-based Posterior Training (PPT) to address this gap directly. The core idea is to use probabilistic programs as a bridge between open-world natural language scenarios and mathematically grounded supervision signals, then fine-tune LLMs on the resulting distributional targets.
The Problem with Standard Approaches
Fine-tuning LLMs for inductive reasoning faces two compounding difficulties. First, collecting high-quality labeled data at scale is expensive; human judgments are slow to gather and hard to calibrate across diverse scenarios. Second, the correct answer to an inductive query is not a point estimate but a probability distribution over latent variables. Standard fine-tuning pipelines, designed around single correct answers, are structurally mismatched to this requirement.
A tempting shortcut is to use a stronger closed-source LLM as a labeler, but this simply propagates whatever biases and overconfidence that model carries. The authors demonstrate empirically that fine-tuning on Gemini-generated labels underperforms PPT, which is a meaningful result given how often LLM-as-judge pipelines are treated as a free lunch.
Methodology: Probabilistic Programs as Supervision
PPT builds on the Model Synthesis Architecture (MSA) from Wong et al. (2025), which uses an LLM to translate natural language scenarios into probabilistic programs. The PPT pipeline extends this in three sequential stages:
- Scenario synthesis: A strong closed-source LLM generates diverse open-world natural language scenarios spanning sports, healthcare, and other domains.
- Program generation: A second LLM pass translates each scenario into a probabilistic program written in Pyro, specifying latent variables, observations, and their generative relationships.
- Posterior inference: The Pyro programs are executed to compute posterior distributions over query variables, producing soft distributional labels rather than point estimates.
These (scenario, query, posterior distribution) tuples are then used to fine-tune Llama-3-8B-Instruct and Qwen-2-7B-Instruct via LoRA. The training signal is the full posterior, not just its mean. This distinction matters: the authors compare posterior distribution targets against forward-sampled mean targets and find that distributional supervision yields better estimation accuracy, while mean-only supervision produces comparable raw calibration but loses its advantage once temperature scaling is applied.
The pipeline generates over 10,000 unique scenarios with more than 50,000 queries, which is a non-trivial scale for a synthetically constructed inductive reasoning dataset with principled probabilistic grounding.
Results and What They Show
Evaluation spans three tiers: held-out synthetic scenarios, human-labeled sports judgments, and seven external benchmarks including MMLU, TruthfulQA, HellaSwag, ARC-C, Winogrande, BayesianTom variants, and OpenEstimate.
Key findings include:
- PPT substantially reduces mean absolute error (MAE) on held-out inductive scenarios relative to base LLMs, including settings with entirely unseen inferential motifs and a held-out healthcare domain not seen during sports-domain training.
- On human-labeled sports scenarios, PPT-fine-tuned models achieve correlation with human judgments comparable to the original MSA probabilistic programs, which is a strong result given that MSA operates at inference time with explicit program synthesis.
- On OpenEstimate, PPT reduces MAE from 27.7 (forward mean baseline) to 23.3, a meaningful improvement on a prior-elicitation benchmark that directly tests calibrated probabilistic estimation.
- Across multiple-choice benchmarks, PPT reduces both negative log-likelihood and expected calibration error (ECE), with gains that persist after post-hoc temperature scaling is applied. This is the key calibration result: the improvements are not simply rescaling outputs but reflect a genuine shift in the model's internal uncertainty representation.
The progressively changing scenario experiment is a nice sanity check. When player strength is incrementally adjusted across eight scenarios, PPT-fine-tuned models track the gradient more faithfully than base LLMs, which plateau prematurely. This suggests the fine-tuned models have internalized something about the relationship between evidence accumulation and posterior updating, rather than pattern-matching to surface features.
Limitations and Open Questions
The approach has real limitations worth taking seriously. The quality of supervision depends entirely on whether the LLM-generated Pyro programs faithfully represent the intended scenario. Probabilistic program synthesis is itself an open research problem, and the authors acknowledge that systematic validation of program quality remains incomplete. A program that misspecifies the prior or the likelihood structure will produce a posterior that is internally consistent but wrong for the scenario it purports to model. Scaling this pipeline while maintaining program fidelity is non-trivial.
The evaluation is also largely confined to numeric and multiple-choice queries, which are well-suited to measuring MAE, NLL, and ECE but do not capture open-ended inductive reasoning. Whether PPT generalises to free-form inference tasks, where the model must generate natural language explanations that reflect calibrated uncertainty, is an open question. Extending the approach to settings with intermediate reasoning steps or multi-turn interactions would be a significant and worthwhile direction.
There is also a question of compute overhead. Running Pyro inference at scale for 50,000+ queries is feasible but not free, and the cost-quality tradeoff relative to alternatives like direct LLM labeling needs more careful analysis as dataset sizes grow.
Broader Implications
The framing this paper offers is worth sitting with. Standard reasoning post-training treats supervision as a signal about correct answers. PPT treats supervision as a signal about correct beliefs, which is a different and arguably more fundamental target for general-purpose reasoning systems. The fact that probabilistic programs can serve as scalable supervision sources, rather than just test-time reasoning tools, opens a direction that has been underexplored relative to the volume of work on prompting and chain-of-thought methods.
The result that distributional soft labels outperform both LLM-generated point labels and fine-tuning on mean estimates reinforces a point that the machine learning community has known for a while in other contexts: training on the full predictive distribution is generally better than training on its mode or mean, and this holds here even when the distribution itself is approximated by a synthesized probabilistic program.
For researchers working on uncertainty quantification, calibration, or the intersection of probabilistic modeling and neural language models, this paper is a concrete and well-executed contribution. The code is available at github.com/zhang-liyi/llm-inductive, and the full paper can be read at arxiv.org/abs/2606.09856.