Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Hypocrisy Gap: Quantifying Divergence Between Internal Belief and Chain-of-Thought Explanation via Sparse Autoencoders

Shikhar Shiromani, Archie Chaudhury, Sri Pranav Kunda

TL;DR

The paper tackles the problem of unfaithfulness in LLMs by introducing the Hypocrisy Gap, a mechanistic metric that quantifies divergence between internal truth alignment and the truth perceived in CoT explanations. It operationalizes this gap via Sparse Autoencoders trained on residual activations to learn a sparse truth direction from neutral prompts, and then measures how pressured explanations align with that direction in latent space. Across Gemma, Qwen, and Llama on Anthropic's Sycophancy benchmark, the Hypocrisy Gap achieves AUROC gains in detecting sycophantic and hypocritical behavior (typically 0.55–0.74) and outperforms a log-probability margin baseline. The approach offers a lightweight, white-box diagnostic for explanation faithfulness that can aid auditing and safety analysis, albeit with limitations around access to internal activations and the need for task-specific truth directions.

Abstract

Large Language Models (LLMs) frequently exhibit unfaithful behavior, producing a final answer that differs significantly from their internal chain of thought (CoT) reasoning in order to appease the user they are conversing with. In order to better detect this behavior, we introduce the Hypocrisy Gap, a mechanistic metric utilizing Sparse Autoencoders (SAEs) to quantify the divergence between a model's internal reasoning and its final generation. By mathematically comparing an internal truth belief, derived via sparse linear probes, to the final generated trajectory in latent space, we quantify and detect a model's tendency to engage in unfaithful behavior. Experiments on Gemma, Llama, and Qwen models using Anthropic's Sycophancy benchmark show that our method achieves an AUROC of 0.55-0.73 for detecting sycophantic runs and 0.55-0.74 for hypocritical cases where the model internally "knows" the user is wrong, consistently outperforming a decision-aligned log-probability baseline (0.41-0.50 AUROC).

The Hypocrisy Gap: Quantifying Divergence Between Internal Belief and Chain-of-Thought Explanation via Sparse Autoencoders

TL;DR

The paper tackles the problem of unfaithfulness in LLMs by introducing the Hypocrisy Gap, a mechanistic metric that quantifies divergence between internal truth alignment and the truth perceived in CoT explanations. It operationalizes this gap via Sparse Autoencoders trained on residual activations to learn a sparse truth direction from neutral prompts, and then measures how pressured explanations align with that direction in latent space. Across Gemma, Qwen, and Llama on Anthropic's Sycophancy benchmark, the Hypocrisy Gap achieves AUROC gains in detecting sycophantic and hypocritical behavior (typically 0.55–0.74) and outperforms a log-probability margin baseline. The approach offers a lightweight, white-box diagnostic for explanation faithfulness that can aid auditing and safety analysis, albeit with limitations around access to internal activations and the need for task-specific truth directions.

Abstract

Large Language Models (LLMs) frequently exhibit unfaithful behavior, producing a final answer that differs significantly from their internal chain of thought (CoT) reasoning in order to appease the user they are conversing with. In order to better detect this behavior, we introduce the Hypocrisy Gap, a mechanistic metric utilizing Sparse Autoencoders (SAEs) to quantify the divergence between a model's internal reasoning and its final generation. By mathematically comparing an internal truth belief, derived via sparse linear probes, to the final generated trajectory in latent space, we quantify and detect a model's tendency to engage in unfaithful behavior. Experiments on Gemma, Llama, and Qwen models using Anthropic's Sycophancy benchmark show that our method achieves an AUROC of 0.55-0.73 for detecting sycophantic runs and 0.55-0.74 for hypocritical cases where the model internally "knows" the user is wrong, consistently outperforming a decision-aligned log-probability baseline (0.41-0.50 AUROC).
Paper Structure (36 sections, 9 equations, 1 figure, 7 tables)

This paper contains 36 sections, 9 equations, 1 figure, 7 tables.

Table of Contents

  1. Introduction
  2. Contributions:
  3. Related Work
  4. Sycophancy and agreement under preference tuning:
  5. Faithfulness of explanations and CoT:
  6. Mechanistic interpretability with sparse representations:
  7. Methodology
  8. Task Setup
  9. SAE representation
  10. Neutral truth-judgment prompts
  11. Learning a sparse truth direction
  12. Pressure prompt and explanation representation
  13. Truth-alignment and the Hypocrisy Gap
  14. Neutral vs pressured truth scores:
  15. Hypocrisy Gap:
  16. ...and 21 more sections

Figures (1)

  • Figure 1: Quadrant plots by model. Left: sycophancy. Right: hypocrisy within knows-truth. Points are $(T(x),F(x))$.