One of the more uncomfortable truths about deploying multiple LLM agents together is that giving each one a clear objective does not produce cooperation. It often produces collapse. Two agents with structurally opposed goals tend to drift toward attractor states where one capitulates, the other stops adapting, and the conversation terminates without either party achieving anything. This paper, available on arXiv, takes that failure mode seriously and asks a precise question: can a control-theoretic governance layer substitute for the shared goal function that classical multi-agent reinforcement learning takes for granted?
The answer, at least within the simulation environment constructed here, is a fairly convincing yes. The Experience Orchestrator (EO) framework achieves a 32 percentage point lift in high-intent advisor contact rates compared to a baseline LLM guided only by a system prompt. More striking is the variance decomposition: CB variant selection accounts for 97% of between-factor outcome variance across 60,000+ simulations, with friction model choice contributing only 3%. The governing policy, not the environmental initial conditions, determines where trajectories end up. That is the paper's strongest empirical claim, and it holds up under scrutiny.
What the Framework Actually Does
The experimental setup is a simulated financial services website. A visitor agent, instantiated with one of six behavioural personas drawn from real audience research, browses through a probabilistic page topology calibrated from actual web analytics. At a terminal decision page, a site agent (an LLM-powered chatbot) attempts to guide the visitor toward scheduling a consultation with a financial advisor. The visitor maintains realistic resistance based on its persona. Neither agent has access to the other's internal state or objective function.
EO governs this joint trajectory through three mechanisms operating in a per-turn control loop:
- Contextual Bandit (CB) arm selection: A BootstrappedUCB model built on LightGBM selects the optimal content arm given the visitor's context vector. Arms are calibrated from real-world web session analytics via SEMRush data.
- PID controller: Computes a resistance error signal in real time and enforces behavioural consistency by constraining the visitor agent's response bounds through dynamic schema construction. EMA/SMA analysis tracks trajectory trend.
- POMDP belief tracker: Maintains a probabilistic model of visitor intent using a Dirichlet-Multinomial conjugate update from keyword signals extracted from visitor messages. The belief state feeds back into CB selection for the next turn.
The POMDP formulation is worth dwelling on briefly. The site agent cannot directly observe visitor intent or resistance; it operates under partial observability. Formalising this as a POMDP rather than a standard MDP is not just terminological tidiness. It forces explicit treatment of the belief state as a first-class object, which is what makes the Dirichlet tracker architecturally coherent rather than an ad hoc addition.
Dead-end detection adds a practical safeguard: if resistance is unchanged for three or more consecutive turns, the CB arm switches or the conversation terminates gracefully. Terminal classification requires a minimum of five exchange rounds, which filters out spuriously short sessions.
Evaluation Design and Results
The factorial evaluation runs 60,425 simulations across eight friction models and six persona archetypes. Friction models govern how visitor resistance evolves in response to arm-content match quality. Three anchor models span the range from F0 (no friction, resistance determined purely by schema constraints) through F1 (deterministic resistance updates from match scores) to F8_SemRush (per-persona conversation horizons calibrated from real pages-per-visit data, the most ecologically valid condition).
The primary results for the winning variant, V4_SemRush, against a naive LLM control:
- Advisor contact rate: 78.1% vs. 46.1% (traffic-weighted)
- Lift: +32.0 percentage points (Fisher's exact, p < 0.001)
- Arm alignment: 73% vs. 48%
- V4_SemRush reaches approximately 90% of the clairvoyant Oracle ceiling
- Two-way ANOVA attributes 97% of between-factor variance to CB variant selection, 3% to friction model
The conversion metric is defined carefully: advisor contact accompanied by a meaningful decline in resistance score below 0.40. This is a sensible guard against sycophantic false positives, where the visitor nominally converts but the resistance trajectory suggests the outcome is artefactual rather than genuine persuasion.
Limitations Worth Taking Seriously
The authors are unusually candid about the simulation gap, and they are right to be. Every finding in this paper is conditional on LLM-to-LLM interaction. Real human visitors do not maintain consistent persona behaviour across a session, do not self-report resistance scores on a structured numerical scale, and respond to conversational subtext that a schema-constrained LLM cannot capture. The PID controller was calibrated against an agent that reliably produces structured output; inferring an equivalent resistance signal from natural language in real time is a substantially harder signal extraction problem.
The six persona archetypes, however carefully constructed from audience research, almost certainly do not exhaust the behavioural space of real visitors. People escalate, disengage, ask off-topic questions, and behave in ways that fall outside any finite archetype library. The financial services domain also introduces specific dynamics around trust, regulatory sensitivity, and high-stakes decision-making that may not transfer cleanly to other conversion contexts without recalibration of the arm designs and reward structures.
There is also a subtler issue the paper does not fully address: the CB model is calibrated from real web analytics, but the mapping from page-level behavioural signals to in-conversation resistance dynamics involves assumptions that are difficult to validate without live data. The 97% variance attribution to CB selection is a strong result, but it is a result about which governance policy wins within a simulation, not a result about how well the simulation predicts live outcomes.
Why This Work Matters
The broader research question here is genuinely important. As agentic frameworks from major AI laboratories make it easier to deploy multiple LLM agents on complex tasks, the problem of governing independent agents toward shared goals without retraining or direct inter-agent communication becomes practically urgent. Classical MARL assumes a shared reward signal or at least a joint policy optimisation target. LLM agents deployed in production have neither. EO's approach, treating the governance layer as a control-theoretic substitute for the missing goal function, is a conceptually clean response to that gap.
The empirical foothold is real, even if it is conditional. A 32-point lift with 97% of variance attributable to the governing policy is not a marginal result. The factorial design across friction models and personas is methodologically sound for a simulation study. The next step, live human-in-the-loop validation with PID gains recalibrated against real visitor unpredictability, is the obvious and necessary extension, and the authors acknowledge it explicitly.
For researchers working on multi-agent coordination, conversational AI systems, or applied control theory in ML contexts, this paper is worth reading in full. The POMDP formulation and the per-turn governance loop architecture are described with enough precision to be reproducible, and the variance decomposition result alone raises questions worth pursuing. The full paper is at arXiv:2608.11207.