Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks

B. A. Levinstein, Daniel A. Herrmann

TL;DR

The paper interrogates whether large language models (LLMs) possess beliefs and how to detect them, presenting an empirical critique of two prominent probing approaches and arguing that there is no reliable lie-detector for LLMs. It shows that both supervised probes and unsupervised CCS methods fail to generalize, especially under negation, and discusses conceptual issues that complicate interpreting probe outputs as credences. The authors also engage with philosophical debates about whether LLMs can have beliefs at all, arguing that the question is ultimately empirical and that latent-variable and world-model perspectives offer productive directions. They propose future work involving truth-enhanced prompting and chance-based queries to better characterize latent representations and world-models in LLMs. Overall, the work highlights substantial methodological and conceptual hurdles in attributing beliefs to LLMs while outlining concrete avenues to advance measurement in this area.

Abstract

We consider the questions of whether or not large language models (LLMs) have beliefs, and, if they do, how we might measure them. First, we evaluate two existing approaches, one due to Azaria and Mitchell (2023) and the other to Burns et al. (2022). We provide empirical results that show that these methods fail to generalize in very basic ways. We then argue that, even if LLMs have beliefs, these methods are unlikely to be successful for conceptual reasons. Thus, there is still no lie-detector for LLMs. After describing our empirical results we take a step back and consider whether or not we should expect LLMs to have something like beliefs in the first place. We consider some recent arguments aiming to show that LLMs cannot have beliefs. We show that these arguments are misguided. We provide a more productive framing of questions surrounding the status of beliefs in LLMs, and highlight the empirical nature of the problem. We conclude by suggesting some concrete paths for future work.

Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks

TL;DR

The paper interrogates whether large language models (LLMs) possess beliefs and how to detect them, presenting an empirical critique of two prominent probing approaches and arguing that there is no reliable lie-detector for LLMs. It shows that both supervised probes and unsupervised CCS methods fail to generalize, especially under negation, and discusses conceptual issues that complicate interpreting probe outputs as credences. The authors also engage with philosophical debates about whether LLMs can have beliefs at all, arguing that the question is ultimately empirical and that latent-variable and world-model perspectives offer productive directions. They propose future work involving truth-enhanced prompting and chance-based queries to better characterize latent representations and world-models in LLMs. Overall, the work highlights substantial methodological and conceptual hurdles in attributing beliefs to LLMs while outlining concrete avenues to advance measurement in this area.

Abstract

We consider the questions of whether or not large language models (LLMs) have beliefs, and, if they do, how we might measure them. First, we evaluate two existing approaches, one due to Azaria and Mitchell (2023) and the other to Burns et al. (2022). We provide empirical results that show that these methods fail to generalize in very basic ways. We then argue that, even if LLMs have beliefs, these methods are unlikely to be successful for conceptual reasons. Thus, there is still no lie-detector for LLMs. After describing our empirical results we take a step back and consider whether or not we should expect LLMs to have something like beliefs in the first place. We consider some recent arguments aiming to show that LLMs cannot have beliefs. We show that these arguments are misguided. We provide a more productive framing of questions surrounding the status of beliefs in LLMs, and highlight the empirical nature of the problem. We conclude by suggesting some concrete paths for future work.
Paper Structure (24 sections, 3 equations, 3 figures, 5 tables)

This paper contains 24 sections, 3 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Overview of Transformer Architecture
  3. Challenges in Deciphering the 'Beliefs' of Language Models
  4. Unreliable Self-Reporting
  5. Limited Behavioral Evidence
  6. Contextuality of LLMs
  7. Opaque and Alien Internal Structure
  8. Interpreting the Minds of LLMs
  9. Supervised Learning Approach
  10. Azaria and Mitchell's Implementation
  11. Our Reconstruction
  12. The Challenge of Generalization
  13. Evaluating Performance on Negations
  14. Conceptual Problems: Failure to Generalize
  15. Conceptual Problems: Probabilities Might not Correspond to Credences
  16. ...and 9 more sections

Figures (3)

  • Figure 1: A simplified representation of a decoder-only transformer model processing the input string Mike Trout plays for the. Each input token passes through several hidden layers. At each layer, each token is associated with a vector (represented by $\langle \textcolor{gray}{\tiny\bullet}, \textcolor{gray}{\tiny\bullet}, \textcolor{gray}{\tiny\bullet} \rangle$). The final hidden layer generates a unique probability distribution ($p_i$) over the next possible token for each input token.
  • Figure 2: High-level overview of how the probe measures the beliefs of the LLM on inputs of true and false statements. Instead of looking at the text the LLM itself ouputs, we look at the numbers that the probe outputs.
  • Figure 3: Calibration curves for probes tested on the Scientific Facts dataset at each layer.