As AI systems take on more autonomous roles, the question of how to align their decisions with human values has moved from philosophy seminar to engineering requirement. Most existing approaches to value detection in text are either tied to a specific theoretical framework or depend heavily on hand-crafted prompts, making them brittle and hard to generalise. This paper proposes a modular, theory-agnostic architecture that separates the conceptualisation of values from their detection, producing a pipeline that can be instantiated with different LLMs and adapted to different value theories without rebuilding from scratch.
The practical motivation is clear: before an autonomous agent can act in a value-aligned way, it needs to know which values a given action promotes or undermines. Static, designer-specified mappings break down when context shifts. Dynamic inference from text is the more tractable path, and this work is a structured attempt to make that inference systematic and reproducible.
Key Contributions
The architecture has three coordinated modules:
- Value Conceptualisation Module (VCM): Takes the foundational texts of any value theory as input and generates structured value specifications. This is the critical piece that makes the system theory-agnostic. Rather than hard-coding Schwartz or Moral Foundations Theory into the prompts, the VCM reads the source material and produces specifications that the rest of the pipeline consumes.
- Value Detection Module (VDM): Uses those specifications to label input texts, identifying which values are present and whether they are being promoted or resisted.
- User Interaction Module (UIM): Provides a human-readable interface for exploring detected values, with support for human-in-the-loop refinement of specifications.
A central orchestrator manages information flow between modules, which elevates the system beyond a collection of prompts into something closer to a coherent automated workflow. The authors draw a useful distinction here: previous multi-stage pipelines like Value Lens formalise a theory and then detect values, but still rely on ad-hoc prompt construction. The VCM approach derives specifications directly from source texts, reducing that manual dependency.
The architecture also introduces a graded intensity scale, assigning each detected value a score reflecting the degree of support or resistance based on rhetorical and semantic evidence. This is a meaningful addition over binary classification, since real-world texts rarely either fully endorse or fully reject a value.
Experimental Setup and Results
Evaluation used the Touché24-ValueEval dataset, a multi-label corpus of 59,662 short texts annotated with Schwartz values. A subset of 7,600 texts was used, balancing statistical reliability against computational cost. The models tested were Gemma3, Llama4-scout, DeepSeek-R1, Qwen3, and a GPT-based open-source model. Performance was measured by micro F1-score, precision, and recall.
The headline result is not which model performed best, but how close they all were:
- Gemma3: micro F1 0.3406, recall 44.8%, precision 27.5%
- Gpt-oss: micro F1 0.3359, recall 33.2%, precision 34.0%
- Llama4-scout: micro F1 0.3275, recall 48.1%, precision 24.8%
- DeepSeek-R1: micro F1 0.3227, recall 30.2%, precision 34.7%
- Qwen3: micro F1 0.3216, recall 27.5%, precision 39.1%
The spread across five architecturally distinct models is less than 0.02 in F1. The authors argue, plausibly, that this tight clustering reflects the dominance of the pipeline's structure over any individual model's capabilities. The structured prompts constrain the output space so heavily that temperature variation has negligible effect, as confirmed by Gemma3 tests at T=0.0 and T=1.0. When the valid next-token set is narrow enough, randomness simply has nowhere to go.
The precision-recall trade-off across models is worth examining. Qwen3 is the most conservative, with the highest precision (39.1%) but lowest recall (27.5%), while Llama4-scout is the most permissive, catching more values but with lower precision. GPT-oss sits in the middle with the most balanced profile. For downstream value alignment applications, this trade-off has real consequences: a high-recall system might flag spurious value conflicts, while a high-precision system might miss genuine ones.
Limitations and Open Questions
The F1 scores in the low 0.3 range are honest but modest. The authors acknowledge this and note that performance is comparable to classical multi-label baselines from the ValueEval shared task, which is a reasonable baseline but not a high bar. Multi-label value detection is genuinely hard, particularly because value categories overlap and texts often carry implicit rather than explicit value signals, but the gap between these results and what would be needed for reliable deployment in autonomous decision systems is still significant.
Several limitations deserve attention:
- The evaluation is conducted entirely on Schwartz's theory. The theory-agnostic claim of the VCM is architecturally plausible but empirically untested across other frameworks. Experiments with Moral Foundations Theory or domain-specific value sets would strengthen the generalisability argument considerably.
- There are no ablation studies. It is not clear how much of the performance comes from the VCM-generated specifications versus what a well-crafted manual prompt would achieve. This is the central empirical question the paper raises but does not answer.
- The intensity scale, which is one of the more interesting contributions, receives little quantitative treatment. Human validation of the graded scores is listed as future work, but without it the scale remains an architectural feature rather than a validated measurement instrument.
- The dataset subset (7,600 of 59,662 texts) is justified on computational grounds, but the sampling strategy and its potential effects on evaluation are not discussed in depth.
The broader implication the authors draw, that principled architectural design matters more than chasing marginal gains in prompt engineering or model selection, is well-supported by their results and worth taking seriously. The field has spent considerable effort on prompt optimisation; this work suggests that investing in structured pipelines may yield more transferable and reproducible gains.
Implications for Value-Aligned AI
The connection to autonomous decision-making is the motivating context throughout, but the architecture itself is evaluated purely as a text classification system. Bridging that gap, integrating this pipeline into an actual decision model and measuring whether value-detection quality translates into better-aligned behaviour, is the obvious and necessary next step. The authors acknowledge this as ongoing work.
For researchers working on value-based argumentation, AI governance, or normative reasoning in multi-agent systems, the modular design is genuinely useful. The separation between conceptualising values and detecting them means that domain experts can update the value specifications without touching the detection machinery, which is a practical advantage in applied settings where value frameworks evolve or need to be localised.
A reference implementation is available at HuggingFace Spaces, which lowers the barrier to replication and extension. Full paper details are at arXiv:2605.27373.