Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Faithful-Patchscopes: Understanding and Mitigating Model Bias in Hidden Representations Explanation of Large Language Models

Xilin Gong, Shu Yang, Zehua Cao, Lynne Billard, Di Wang

TL;DR

Faithful-Patchscopes reveals that Patchscopes explanations are often unfaithful due to inherent linguistic biases in LLMs. The authors propose BALOR, a logit-recalibration decoding method using a contrastive prompt to suppress bias and amplify context, with theoretical guarantees on log-odds as the amplification factor grows. They design a bias-focused Q&A dataset and show BALOR consistently improves faithfulness across multiple models and tasks, achieving up to substantial relative gains while remaining robust to hyperparameters and temperature. The work offers a practical, inference-time solution for more faithful hidden-representation explanations, contributing to more trustworthy interpretability tools in large language models.

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities for hidden representation interpretation through Patchscopes, a framework that uses LLMs themselves to generate human-readable explanations by decoding from internal hidden representations. However, our work shows that LLMs tend to rely on inherent linguistic patterns, which can override contextual information encoded in the hidden representations during decoding. For example, even when a hidden representation encodes the contextual attribute "purple" for "broccoli", LLMs still generate "green" in their explanations, reflecting a strong prior association. This behavior reveals a systematic unfaithfulness in Patchscopes. To systematically study this issue, we first designed a dataset to evaluate the faithfulness of Patchscopes under biased cases, and our results show that there is an 18.84\% faithfulness decrease on average. We then propose Bias Alignment through Logit Recalibration (BALOR), which treats the output logits from an unpatched prompt as capturing model bias and contrasts them with logits obtained under patched contextual information. By recalibrating the logit distribution through this contrast, BALOR suppresses model bias and amplifies contextual information during generation. Experiments across multiple LLMs demonstrate that BALOR consistently outperforms existing baselines, achieving up to 33\% relative performance improvement.

Faithful-Patchscopes: Understanding and Mitigating Model Bias in Hidden Representations Explanation of Large Language Models

TL;DR

Faithful-Patchscopes reveals that Patchscopes explanations are often unfaithful due to inherent linguistic biases in LLMs. The authors propose BALOR, a logit-recalibration decoding method using a contrastive prompt to suppress bias and amplify context, with theoretical guarantees on log-odds as the amplification factor grows. They design a bias-focused Q&A dataset and show BALOR consistently improves faithfulness across multiple models and tasks, achieving up to substantial relative gains while remaining robust to hyperparameters and temperature. The work offers a practical, inference-time solution for more faithful hidden-representation explanations, contributing to more trustworthy interpretability tools in large language models.

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities for hidden representation interpretation through Patchscopes, a framework that uses LLMs themselves to generate human-readable explanations by decoding from internal hidden representations. However, our work shows that LLMs tend to rely on inherent linguistic patterns, which can override contextual information encoded in the hidden representations during decoding. For example, even when a hidden representation encodes the contextual attribute "purple" for "broccoli", LLMs still generate "green" in their explanations, reflecting a strong prior association. This behavior reveals a systematic unfaithfulness in Patchscopes. To systematically study this issue, we first designed a dataset to evaluate the faithfulness of Patchscopes under biased cases, and our results show that there is an 18.84\% faithfulness decrease on average. We then propose Bias Alignment through Logit Recalibration (BALOR), which treats the output logits from an unpatched prompt as capturing model bias and contrasts them with logits obtained under patched contextual information. By recalibrating the logit distribution through this contrast, BALOR suppresses model bias and amplifies contextual information during generation. Experiments across multiple LLMs demonstrate that BALOR consistently outperforms existing baselines, achieving up to 33\% relative performance improvement.
Paper Structure (40 sections, 1 theorem, 17 equations, 9 figures, 10 tables)

This paper contains 40 sections, 1 theorem, 17 equations, 9 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Inherit Model Bias will Affect Explanation Faithfulness
  4. Dataset
  5. Experiment Settings
  6. Analysis
  7. Methodology
  8. Locate the Bias-sensitive Layer
  9. BALOR
  10. Experiments
  11. Settings
  12. Main result analysis
  13. Ablation Study
  14. Related Work
  15. Conclusion
  16. ...and 25 more sections

Key Result

Theorem 4.1

Let $l_\theta(y\mid T)$ and $l_\theta(y\mid T^*)$ be the (pre-softmax) logits over vocabulary $\mathcal{V}$. Define BALOR as Then, for any two tokens $y_1,y_2\in\mathcal{V}$, where $p_\theta(\cdot\mid T)=\sigma(l_\theta(\cdot\mid T))$ and $p_\theta(\cdot\mid T^*)=\sigma(l_\theta(\cdot\mid T^*))$. Equivalently, In particular, if $y_1$ is more supported by the patched target evidence than by the

Figures (9)

  • Figure 1: Overview of the Patchscopes framework and an example illustrating its unfaithfulness under an imbalanced linguistic pattern. The patched hidden representation encodes the contextual attribute "purple" for "broccoli", but under Patchscopes' framework, LLMs still generate "green" in their explanations.
  • Figure 2: The result of attribute SR for vanilla Patchscopes on biased and non-biased datasets.
  • Figure 3: Isotonic Regression of GSA and LD on Llama3.2-1b
  • Figure 4: The selected best layer of each model for patching. Layer index starts from 1 with $w$=0.8.
  • Figure 5: An overview of BALOR method. We use the same example with Figure \ref{['fig:introexample']}, while green is a biased attribute value and purple is a correct attribute value. With the recalibration approach between the logit distributions of contrastive prompt and target prompt in BALOR process, the LLM provides a correct response.
  • ...and 4 more figures

Theorems & Definitions (1)

  • Theorem 4.1: Log-odds contrastive amplification of BALOR