On the Universal Truthfulness Hyperplane Inside LLMs

TL;DR

The paper investigates whether a universal truthfulness hyperplane exists inside large language models by training linear probes on a diverse set of hallucination-detection tasks. Using LR and MM probes applied to attention-head representations extracted from the last input token, it demonstrates that diversity in training data markedly improves cross-task, cross-domain, and in-domain generalization, achieving roughly 70% cross-task accuracy and outperforming strong baselines. While finetuning still yields the highest performance, the probes provide interpretability about true internal representations of truthfulness and show data-efficient learning, with as few as 10 samples per dataset often sufficing. Overall, the work offers evidence that a universal, linearly separable truthfulness signal may reside in LLM hidden states, with broad implications for robust factuality and future interventions.

Abstract

While large language models (LLMs) have demonstrated remarkable abilities across various fields, hallucination remains a significant challenge. Recent studies have explored hallucinations through the lens of internal representations, proposing mechanisms to decipher LLMs' adherence to facts. However, these approaches often fail to generalize to out-of-distribution data, leading to concerns about whether internal representation patterns reflect fundamental factual awareness, or only overfit spurious correlations on the specific datasets. In this work, we investigate whether a universal truthfulness hyperplane that distinguishes the model's factually correct and incorrect outputs exists within the model. To this end, we scale up the number of training datasets and conduct an extensive evaluation -- we train the truthfulness hyperplane on a diverse collection of over 40 datasets and examine its cross-task, cross-domain, and in-domain generalization. Our results indicate that increasing the diversity of the training datasets significantly enhances the performance in all scenarios, while the volume of data samples plays a less critical role. This finding supports the optimistic hypothesis that a universal truthfulness hyperplane may indeed exist within the model, offering promising directions for future research.

Figures (7)

• Figure 1: Top: we extract representations from the last token of the input sequence, then specific locations of the hidden states inside the LLM are selected and concatenated as input to train the probe. Bottom: Previous works mainly train the linear probe on one dataset which tends to overfit spurious features. Our work utilizes diverse datasets to examine whether a universal truthfulness hyperplane exists that can generalize to out-of-domain data.
• Figure 2: Our curated datasets and tasks. Left (Blue) part represents the training tasks, while the right (Orange) represents the test tasks.
• Figure 3: Examples of sparsity test on different datasets using the logistic regression (LR) and the mass mean (MM) probe.
• Figure 4: The analysis experiment results of training on attention head and layer activations, scaling number of training tasks, and varying training split size per task.
• Figure 5: Ablation study of varying $num$ on cross-task test, where $k$ is the compression hyperparameter (128 represents all dimensions of the attention head output).