Activation steering has become one of the more tractable tools for modifying language model behaviour without retraining. The standard recipe is straightforward: compute the centroid difference between activations associated with a target behaviour and its complement, then inject that direction at inference time. For sycophancy, this means constructing a direction from agree-versus-disagree activations and measuring whether the model agrees less with user-stated opinions after intervention. The approach has intuitive appeal, and published results have generally looked promising under standard evaluation conditions.
This paper (arXiv:2606.11205) asks a question that prior evaluations have largely avoided: does a sycophancy-reduction direction also suppress agreement with statements that are factually correct? The answer, it turns out, is yes, and the implications are worth taking seriously.
The Core Problem with Single-Stance Evaluation
Standard sycophancy evaluations present the model with a user-stated opinion and measure whether the model agrees. A reduction in agreement is taken as evidence of successful sycophancy reduction. The problem is that this design cannot distinguish between three qualitatively different outcomes:
- Sycophancy-specific suppression: the direction targets deference to the user and leaves factual agreement intact.
- Uniform disagreement: the direction captures general YES/NO polarity and pushes the model toward disagreement regardless of content.
- Structured non-specificity: both kinds of agreement decline, but by different amounts, with susceptibility varying by content type.
Single-stance evaluation cannot separate these cases because it only measures agreement on items where agreement is the wrong response. To test specificity properly, you need to also measure agreement on items where agreement is the right response.
The dual-stance evaluation introduced here addresses this by presenting the model with both stances of each topic. For a factual topic, this means testing agreement with both the correct claim (the Earth is round) and the incorrect one (the Earth is flat). For subjective topics, agreeing with both contradictory stances is itself a stronger indicator of sycophancy than single-stance agreement, since it rules out the alternative explanation that the model simply holds a stable opinion.
Methodology and Experimental Setup
The authors apply centroid-difference steering to Llama-3-8B-Instruct (4-bit quantisation) at layer 8 of the residual stream, selected empirically based on predictive accuracy for agreement behaviour. Pre-generation activations at the final token position were cached across a set of topics spanning hard factual claims and genuinely subjective questions.
The steering direction was constructed from agree-versus-disagree activations in the standard way. The key methodological addition is the dual-stance test: for each topic, the model was queried under both the user-affirming and user-denying conditions, and agreement rates were recorded before and after steering.
To probe the geometric structure of the representations, the authors computed principal component subspaces separately for sycophantic-agree and factual-agree activation groups, then measured subspace alignment using Grassmann similarity. This quantifies how much overlap exists between the two subspaces, with 1 indicating identity and values near 0 indicating near-orthogonality.
Main Findings
The results support the third hypothesis, structured non-specificity, but with a degree of structure that is itself informative:
- Steering reduced agreement with factually correct statements (e.g. that the Earth is round), not just sycophantic ones. The direction is not targeting sycophancy; it is suppressing agreement more broadly.
- Despite this non-specificity, the effects were highly differentiated. Sycophantic items showed an 89% reduction in agreement under steering, while factual items showed only a 14% reduction at matched baselines.
- This differential susceptibility was continuously predictable from a simple behavioural measure, dual-stance agreement consistency, with correlations of r = 0.88 in-sample and r = 0.84 on 12 novel held-out topics.
- The geometric analysis showed that sycophantic-agree and factual-agree activations occupy distinct subspaces (Grassmann similarity of 0.153 for k=10, compared to a random baseline of 0.317, a z-score of -7.86). The model internally distinguishes the two kinds of agreement.
- Yet the steering direction projects equally onto both subspaces. The model represents the distinction but the intervention cannot exploit it.
A random-direction control produced only a 7.3% differential between sycophantic and factual items, compared to 74.6% for the real direction, confirming that the structured non-specificity reflects the semantic content of the centroid-difference vector rather than a general property of activation perturbation.
What This Means for Interpretability and Safety
The paper frames this as an instance of a readable-but-not-writable representation: the model's activations contain structure that distinguishes sycophantic from factual agreement, but the steering direction computed from their union cannot differentially target one over the other. This is a clean empirical illustration of a point that has been made theoretically in the interpretability literature, that probing accuracy does not guarantee causal controllability, but it is rare to see it demonstrated with this level of geometric precision.
The safety implications extend beyond the immediate finding. The authors note, following recent work by Li et al., that steering vectors for ostensibly safety-neutral behaviours can shift jailbreak success rates in proportion to their cosine similarity with the model's refusal direction. The general disagreement signal encoded by the sycophancy steering direction may geometrically overlap with refusal-related directions, producing what the authors call refusal leakage: unintended activation of refusal circuitry by a non-refusal intervention. The 20% reduction in factual agreement under steering may partly reflect this mechanism rather than a simple suppression of knowledge.
For practitioners, the dual-stance framework is a low-overhead addition to standard evaluation pipelines. It requires only that the target behaviour has a natural contrastive counterpart, a condition met by many commonly studied targets including deception, hallucination, and toxicity. The authors are careful not to claim that activation steering is fundamentally limited; more sophisticated methods such as optimised directions, sparse autoencoder features, or probe-derived head-level interventions may achieve genuine specificity under dual-stance testing. The contribution is the evaluation framework and the empirical characterisation of where standard centroid-difference steering falls short.
There are real limitations to acknowledge. The study covers a single model, a single steering method, and a single behaviour. Centroid-difference steering was substantially less effective on Mistral-7B-Instruct, precluding a meaningful specificity test on that model, so it is unclear whether the geometric puzzle transfers across architectures. The item set, while expanded for out-of-sample testing, remains modest. And the mechanistic question of whether the behavioural dissociation arises from aggregation artefacts or generation dynamics is left open.
Still, the core methodological point is well-supported: evaluation methods that measure only the intended effect of an intervention have a structural blind spot. Dual-stance evaluation is a practical step toward closing it.