Long-form creative writing has quietly become one of the more revealing stress tests for language models. Generating a coherent, engaging story of several thousand words requires sustaining narrative structure, character consistency, and stylistic quality across a much longer context than most benchmark tasks demand. Frontier models handle this reasonably well; smaller open-weight models largely do not. They either truncate early, failing to meet requested lengths, or they produce text that degrades visibly in quality as length increases. POLARIS, from researchers at UMass Amherst and Google, attacks this problem directly with a training recipe designed to be practical on limited compute.
The Core Problem and Why It's Hard
The difficulty here is not simply one of context window size. Many 9B-class models can technically attend over long sequences. The problem is behavioural: models trained primarily on instruction-following and short-form tasks have no strong incentive to produce sustained, high-quality long-form output. Standard supervised fine-tuning on story data helps somewhat, but it doesn't directly optimise for the properties that make a story good, nor does it reward length adherence in a principled way.
Reinforcement learning from human feedback (RLHF) and its variants offer a path forward, but scaling reward models or collecting large volumes of human preference data for creative writing is expensive. The POLARIS approach sidesteps the most expensive parts of that pipeline with two specific design choices that are worth examining carefully.
Key Contributions and Methodology
The system is built on GRPO (Group Relative Policy Optimisation), a variant of policy gradient training that avoids the need for a separate critic model by computing advantages relative to a group of sampled outputs. The two ingredients the authors add on top of standard GRPO are what distinguish POLARIS:
- Frontier LLM judge with a structured rubric: Rather than training a reward model from scratch, the authors use a capable frontier model as an online judge, scoring generated stories against a structured Story Quality rubric covering dimensions like narrative coherence, character development, prose quality, and length adherence. This is an increasingly common approach, but the structured rubric matters here because open-ended creative quality is notoriously hard to score reliably without explicit criteria.
- Human Reference Injection (HRI): Within each GRPO group, the authors include a teacher-forced human-written story as a high-reward anchor. The intuition is that GRPO computes relative advantages across the group, so including a genuinely good human example raises the ceiling and provides a consistent positive signal. This is an elegant and low-cost way to incorporate human-quality references without full supervised fine-tuning on those examples.
The training data consists of roughly 1,400 prompt-story pairs derived from 100 short-story anthologies, which is a deliberately small dataset. Training runs on 4 A100 GPUs, making this accessible to academic labs rather than requiring industrial-scale infrastructure. The base model is Qwen3.5-9B, and the resulting system is POLARIS-9B.
Results and Length Generalisation
The evaluation spans five benchmarks covering both in-distribution and out-of-distribution prompts and rubrics. The headline results are:
- POLARIS-9B is competitive with open-weight models significantly larger than itself across automatic metrics.
- A blinded human evaluation finds POLARIS-9B preferred over the base Qwen3.5-9B and roughly on par with Qwen3.5-27B, a model three times its size.
- POLARIS-9B follows length instructions more closely than comparable models.
The most interesting finding concerns length generalisation. The model is trained exclusively on stories up to 4,000 words, yet it preserves quality on prompts requesting stories up to 12,000 words. Most open-weight models at this scale degrade substantially in this regime, either in quality, length adherence, or both. The fact that POLARIS-9B generalises beyond its training distribution in this specific dimension suggests the training recipe is instilling something more than simple pattern matching on story length.
The authors also make a broader methodological point that deserves attention: length generalisation may be a more discriminating benchmark for creative writing models than standard quality metrics alone. Two models that score similarly on a 2,000-word story task can diverge sharply when asked for 10,000 words. This framing is useful for the field and likely to influence how future creative writing systems are evaluated.
Limitations and Open Questions
Several questions are worth raising. The reliance on a frontier LLM judge introduces a dependency that has known failure modes. LLM judges can be inconsistent on subjective creative quality, may exhibit biases toward certain styles, and can be gamed by outputs that superficially match rubric criteria without genuine quality. The structured rubric mitigates some of this, but the degree to which judge reliability affects training stability is not fully characterised here.
The training dataset of 1,400 stories is small, and all derived from short-story anthologies. This raises questions about genre coverage and whether the model generalises well to creative writing styles not well represented in that corpus. The out-of-distribution benchmark results are encouraging, but the distribution shift from anthology fiction to, say, genre fiction or experimental prose may be larger than the benchmarks capture.
The human evaluation, while blinded, compares POLARIS-9B to Qwen3.5-27B rather than to frontier models like GPT-4o or Claude. The gap between a well-trained 9B model and actual frontier models on long-form creative writing is likely still substantial, and situating POLARIS-9B in that broader context would strengthen the claims.
There is also the question of what the HRI mechanism is actually doing. The authors frame it as providing a high-reward anchor, but it could alternatively be interpreted as a form of implicit supervised fine-tuning signal smuggled into the RL training loop. Ablations on HRI are presumably in the paper, but understanding whether the benefit comes from the anchor's quality, its presence as a reference point for relative advantage computation, or simply increased data diversity would be valuable.
Despite these caveats, POLARIS represents a practically significant contribution. The combination of a small, high-quality dataset, a structured judge, and human reference injection achieves results that previously required either much larger models or much more compute. For researchers working on specialised creative writing applications with limited resources, the recipe is directly applicable. The length generalisation finding also opens a productive direction for evaluation methodology that the field should take seriously.
The paper is available at arXiv:2606.04095.