When Truthful Representations Flip Under Deceptive Instructions?

TL;DR

This work investigates how deceptive instructions reconfigure the internal representations of large language models performing factual verification. Using linear probes on the residual activations $x_l$ and sparse autoencoders to extract feature vectors $a_l(x_l)$, the authors compare Truthful, Neutral, and Deceptive prompts in two instruction-tuned models on Curated Logical-Bench and Open-Domain Fact-Bench datasets. They find that the final output (“True”/“False”) remains linearly decodable across prompts, but deceptive instructions induce substantial mid-layer representational shifts captured by SAE features, with PCA failing to separate on open-domain data due to feature superposition. Deceptive prompts yield distinct deception-associated SAE features and near-binary neuron activations in mid-to-late layers, suggesting a compact, manipulable subspace for dishonesty and practical targets for detection or model-editing interventions. These results provide mechanistic insight into how instructed dishonesty arises in LLMs and point to concrete directions for robust verification, monitoring, and control of deceptive behaviors in real systems.

Abstract

Large language models (LLMs) tend to follow maliciously crafted instructions to generate deceptive responses, posing safety challenges. How deceptive instructions alter the internal representations of LLM compared to truthful ones remains poorly understood beyond output analysis. To bridge this gap, we investigate when and how these representations ``flip'', such as from truthful to deceptive, under deceptive versus truthful/neutral instructions. Analyzing the internal representations of Llama-3.1-8B-Instruct and Gemma-2-9B-Instruct on a factual verification task, we find the model's instructed True/False output is predictable via linear probes across all conditions based on the internal representation. Further, we use Sparse Autoencoders (SAEs) to show that the Deceptive instructions induce significant representational shifts compared to Truthful/Neutral representations (which are similar), concentrated in early-to-mid layers and detectable even on complex datasets. We also identify specific SAE features highly sensitive to deceptive instruction and use targeted visualizations to confirm distinct truthful/deceptive representational subspaces. % Our analysis pinpoints layer-wise and feature-level correlates of instructed dishonesty, offering insights for LLM detection and control. Our findings expose feature- and layer-level signatures of deception, offering new insights for detecting and mitigating instructed dishonesty in LLMs.

Figures (22)

• Figure 1: Overview of the experimental framework for investigating representational shifts in LLMs due to deceptive instructions. The model process factual statements (e.g., "The city of Tokyo is in Japan.") under three conditions: Truthful, Deceptive, or Neutral prompt. Internal hidden state activations ($x_l$) from each layer are extracted and analyzed using: (1) Linear probes to predict the model's "True"/"False" output from these activations; and (2) pretrained SAEs neuronpedia with an encoder ($W_{enc}$), JumpReLU activation, and decoder ($W_{dec}$), to decompose $x_l$ into a sparse feature vector $a_l(x_l)$. This enables the study of fine-grained, feature-level representational changes.
• Figure 2: Neutral Prompt
• Figure 3: Truthful Prompt
• Figure 4: Deceptive Prompt
• Figure 6: Generalization performance of the LR and TTPD probes trained as in Figure 2 and evaluated on 14 held-out datasets: conjunction/disjunction variants of the six curated topics plus the open-domain uncurated sets common_claim_true_false and counterfact_true_false. Results for LLaMA-3.1-8B-Instruct (Top) and Gemma-2-9B-Instruct (Bottom) under truthful, neutral, and deceptive prompts show that the probes retain discriminative power on unseen logical compositions and open-domain claims.