Generating a physics diagram is not the same as generating a picture of a physics scenario. Every arrow in a free-body diagram is a formal claim: a force vector pointing in the wrong direction is not an aesthetic failure, it is a false statement about the physical world. Current generative models, including state-of-the-art diffusion systems and frontier vision-language models, treat diagram generation as an image synthesis problem and consistently fail at this distinction. They hallucinate force directions, violate conservation laws, and produce geometrically inconsistent angle relationships that look plausible but are physically wrong.
PhyDrawGen addresses this directly by separating the semantic task of understanding a physics problem from the symbolic task of satisfying physical constraints. The paper is worth attention not just for its benchmark results, but because the architectural decision it makes, decoupling probabilistic language understanding from deterministic constraint solving, is a principled response to a class of failure modes that scaling alone is unlikely to fix.
Key Contributions
The paper makes three concrete contributions:
- A typed scene graph schema covering mechanics, optics, and electromagnetism under a unified vocabulary of six node classes and six edge relations, where Constraint nodes map directly to algebraic physical conditions.
- A deterministic PSLG (Planar Straight-Line Graph) constraint solver that converts the scene graph into exact geometric primitives encoding force balance, optical ray consistency, and field line topology.
- A propose-verify correction loop using a fine-tuned Qwen-VL model that iteratively checks constraint violations against the symbolic solver and corrects them, up to a maximum of five iterations.
The benchmark itself is also a contribution: 1,449 problems spanning mechanics, optics, and electromagnetism, including a 65-problem open-vocabulary subset where physical objects are arbitrary real-world entities rather than standard physics apparatus.
Methodology
The pipeline runs in three stages. First, a large language model extracts a typed scene graph from the problem text. This is not a novel idea in isolation; scene graph extraction from natural language is well-established. The novelty is in what happens next. The scene graph feeds into a deterministic analytical solver that produces a PSLG, where force balance, surface normals, friction directions, optical paths, and field topologies are encoded as exact geometric constraints rather than learned approximations. The solver does not hallucinate because it has no learned prior to hallucinate from.
The third stage addresses the residual gap between the symbolic output and a renderable diagram. A Qwen-VL model fine-tuned on automatically generated constraint-violation examples checks the rendered output against the symbolic ground truth and proposes corrections. This is a sensible division of labour: the symbolic solver handles what it can guarantee exactly, and the vision-language model handles the visual grounding that the solver cannot.
The evaluation methodology is careful about independence. Ground-truth arrows are derived from the GPT-extracted scene graph and problem text using closed-form textbook formulas, independently of the constraint solver. This matters because a system that optimises for its own internal consistency rather than external physical truth would otherwise inflate its scores.
Results
On standard textbook problems, PhyDrawGen achieves a mean angular error of 0.4 degrees against the next-best baseline at 2.2 degrees from GPT-5-image. That is not a marginal improvement; it reflects the difference between approximate visual plausibility and geometric exactness. The VLM-judged constraint satisfaction rate of 94.7% and labelled constraint satisfaction rate of 77.9% both lead all baselines by substantial margins.
The open-vocabulary results are arguably more interesting. When physical objects are elephants, penguins, or cyclists rather than standard blocks and inclines, purely neural systems lose whatever implicit geometric prior they had about where forces attach. PhyDrawGen's VCSR of 92.3% against GPT-5-image's 46.2% on this subset confirms that the architectural claim holds: force geometry is derived from the scene graph's action and contact edges, not from object identity. The one place where PhyDrawGen trails is the blind judge score on open-vocabulary problems, where Gemini 3 Pro's 55.2% edges out PhyDrawGen's 53.6%. This reflects the latter's richer visual rendering of unusual objects, not its physical correctness.
Human judge results reinforce the automated metrics. In forced-choice comparisons, human physics readers preferred PhyDrawGen outputs at rates of 65.3% for mechanics and 69% for electromagnetism, with the gap narrowing to 48.9% on open-vocabulary problems, consistent with the visual rendering limitation noted above.
Per-force-type breakdown shows 100% accuracy on gravity and spring forces, where the PSLG's explicit enforcement of vertical gravity and surface-perpendicular normals gives it a structural advantage. Tension and applied forces score lower across all methods, suggesting these are harder to ground symbolically from text descriptions.
Limitations and Implications
The authors are honest about the boundaries of the approach. The PSLG is inherently 2D and planar, which excludes 3D force configurations and any domain where topology cannot be reduced to a planar graph. The benchmark is drawn from early-undergraduate curricula; Olympiad-level mechanics and quantum phenomena such as Feynman diagrams are explicitly out of scope. The pipeline's dependence on an initial LLM extraction means that underspecified problem text can produce structural omissions that the correction loop, capped at five iterations, cannot always resolve. If the extractor misreads a surface angle, the solver will be consistent but wrong.
There is also a methodological point worth raising. The baseline comparisons are restricted to proprietary frontier models, which the authors justify on the grounds that open-weight models lack the zero-shot compositional capability needed for meaningful comparison. This is probably correct, but it means the results are benchmarked against a moving target. GPT-5-image and Gemini 3 Pro will be superseded, and it is not obvious that the margin will hold as purely neural systems continue to scale.
The deeper question the paper raises is whether hard constraint satisfaction is a capability that can be learned at all, or whether it requires a symbolic component by construction. The evidence here suggests the latter, at least for domains where physical laws are algebraically exact and violations are unambiguous. The neuro-symbolic approach taken here is not novel as a category, but the specific instantiation, using a PSLG as the constraint representation and a fine-tuned VLM as the visual correction mechanism, is a concrete and well-evaluated architecture for a problem that matters for science education and automated reasoning systems alike.
Full details are available at arXiv:2605.30512.