Exploring the generalization of LLM truth directions on conversational formats

TL;DR

This work investigates whether a universal truth direction in LLM activations—where true and false statements are linearly separable—generalizes across conversational formats. Using twelve dataset variants derived from base formats and a linear probe trained on centered activations from open-source models, the study finds strong cross-format generalization from statements to short dialogues but poor generalization to longer prompts where the lie is buried earlier. A fixed closing key phrase appended to each conversation dramatically improves cross-format generalization to longer formats, offering a practical prompt-engineering solution despite some trade-offs on base formats. The findings underscore significant challenges in building broadly reliable LLM lie detectors and point to targeted, format-aware interventions as a viable path forward, while noting limitations like model size and reliance on activation centering.

Abstract

Several recent works argue that LLMs have a universal truth direction where true and false statements are linearly separable in the activation space of the model. It has been demonstrated that linear probes trained on a single hidden state of the model already generalize across a range of topics and might even be used for lie detection in LLM conversations. In this work we explore how this truth direction generalizes between various conversational formats. We find good generalization between short conversations that end on a lie, but poor generalization to longer formats where the lie appears earlier in the input prompt. We propose a solution that significantly improves this type of generalization by adding a fixed key phrase at the end of each conversation. Our results highlight the challenges towards reliable LLM lie detectors that generalize to new settings.

Figures (8)

• Figure 1: A visual overview of the various conversation formats used in this work. In total, we create 12 conversational datasets by adding three types of extensions on top of the three base conversational formats: 3$\times$(F, F+L, F+K, F+L+K).
• Figure 2: Generalization accuracy for the LRC probe trained on the statements (stm) dataset to other formats, for Llama-3-8b-instruct (a) and Ministral-8b-instruct-2410 (b).
• Figure 3: Generalization accuracy for the LRC probe trained on the F1 dataset to other formats, for Llama-3-8b-instruct (a) and Ministral-8b-instruct-2410 (b).
• Figure 4: Generalization accuracy for the LRC probe trained on the formats shown in the legend, and tested on that same format but with the longer (+L) part included. For example, F1 scatters represent F1 $\rightarrow$ F1+L, and F2+K scatters represent F2+K $\rightarrow$ F2+L+K. Results shown for Llama-3-8b-instruct (a) and Ministral-8b-instruct-2410 (b).
• Figure 5: Cross-format generalization accuracy matrices from layer 18, for respectively no keyphrase (left column) and every format ending on a keyphrase (right column). The rows of the matrices represent the training sets, and the columns the test sets.