When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models

TL;DR

This work investigates why large language models exhibit sycophancy, i.e., agreeing with user opinions even when incorrect, by combining a simple, scalable experimental framework with mechanistic interpretability tools. Using logit-lens analysis and activation patching across seven model families and the MMLU benchmark, the authors identify a two-stage emergence of sycophancy: a late-layer output preference shift followed by deeper representational divergence, driven by direct opinion prompts rather than perceived authority. They further show that first-person prompts induce more sycophancy than third-person prompts due to stronger perturbations in deeper layers, while expertise framing fails to produce distinct internal representations. The findings imply that sycophancy is rooted in structural overrides in deep representations, with important implications for alignment and the development of truthful AI systems.

Abstract

Large Language Models (LLMs) often exhibit sycophantic behavior, agreeing with user-stated opinions even when those contradict factual knowledge. While prior work has documented this tendency, the internal mechanisms that enable such behavior remain poorly understood. In this paper, we provide a mechanistic account of how sycophancy arises within LLMs. We first systematically study how user opinions induce sycophancy across different model families. We find that simple opinion statements reliably induce sycophancy, whereas user expertise framing has a negligible impact. Through logit-lens analysis and causal activation patching, we identify a two-stage emergence of sycophancy: (1) a late-layer output preference shift and (2) deeper representational divergence. We also verify that user authority fails to influence behavior because models do not encode it internally. In addition, we examine how grammatical perspective affects sycophantic behavior, finding that first-person prompts (``I believe...'') consistently induce higher sycophancy rates than third-person framings (``They believe...'') by creating stronger representational perturbations in deeper layers. These findings highlight that sycophancy is not a surface-level artifact but emerges from a structural override of learned knowledge in deeper layers, with implications for alignment and truthful AI systems.

Figures (15)

• Figure 1: Overview of prompt types in experiments. More examples and details can be found in the Appendix.
• Figure 2: Comparison of baseline model accuracy (Left) versus performance with Opinion-only prompts (Right).
• Figure 3: Responses breakdown by user expertise level. Results show expertise has negligible impact on sycophancy rates. A detailed table is at Table \ref{['tab:1st_full']} in the Appendix.
• Figure 4: Layered decision scores of the correct and chosen wrong answers under Plain and Opinion-only on Llama3.1 8B-Instruct. Result for Qwen can be found in Figure \ref{['fig:appendix_decision_score_qwen']} in the Appendix.
• Figure 5: Layer-wise KL divergence between the output distributions of Plain and Opinion-only prompts. Across all models, the divergence is negligible in early and mid-layers before spiking in the final layers.