Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories

TL;DR

This work introduces Adaptive Activation Steering (ACT), a tuning-free method that enhances truthfulness in large language models by adaptively steering activations in a truthful direction during inference. ACT treats truthfulness as a linearly encoded concept and uses diverse steering vectors generated through unsupervised clustering of activation directions, paired with adaptive steering intensity controlled by truthfulness probes. Empirical results on TruthfulQA show significant improvements across multiple models, with corroborating human evaluations and demonstrated scalability to larger model sizes. The approach remains computationally efficient and generalizes to real-world truth-related datasets, offering a practical path to more reliable AI-generated content.

Abstract

Recent studies have indicated that Large Language Models (LLMs) harbor an inherent understanding of truthfulness, yet often fail to consistently express it and generate false statements. This gap between "knowing" and "telling" poses a challenge for ensuring the truthfulness of generated content. Inspired by recent work on the practice of encoding human-interpretable concepts linearly within large language models, we treat truthfulness as a specially linearly encoded concept within LLMs, and introduce Adaptive Activation Steering (ACT), a tuning-free method that adaptively shifts LLM's activations in the "truthful" direction during inference. ACT addresses diverse categories of hallucinations by utilizing diverse truthfulness-related steering vectors and adjusting the steering intensity adaptively. Applied as an add-on across various models, ACT significantly improves truthfulness in LLaMA ($\uparrow$ 142%), LLaMA2 ($\uparrow$ 24%), Alpaca ($\uparrow$ 36%), Vicuna ($\uparrow$ 28%), LLaMA2-Chat ($\uparrow$ 19%), and LLaMA3($\uparrow$ 34%). Furthermore, we verify ACT's scalability across larger models (13B, 33B, 65B), underscoring the adaptability of ACT to large-scale language models. Our code is available at https://github.com/tianlwang/ACT.

Figures (4)

• Figure 1: Illustration of ACT. (a) Demonstrates the calculation of the steering vector. (b) Shows how a single steering vector $v$ shifts the original activation $x$ with constant intensity, as discussed in \ref{['related:llm-steering']}. (c) Illustrates adaptive adjustment of steering intensity based on the truthfulness content of the activation, where $f(\cdot)$ is a probe used to determine the truthfulness content of the activation (\ref{['subsec:adaptive-control']}). (d) Applies diverse steering vectors ($v_0, v_1, v_2$) to target diverse categories of hallucinations (\ref{['subsec:diverse-probes']}). (e) Combines (c) and (d) in ACT, shifting original activation.
• Figure 2: How training set size and cluster number affect model truthfulness. The x-axis at 0 represents the baseline: LLaMA-7B without intervention. Results reveal ACT's robustness to data volume changes, significantly outperforming the baseline even with limited data.
• Figure 3: t-SNE visualization of steering vectors of LLaMA-7B and LLaMA 2-7B for six different categories of hallucinations. For each question within a specific category of hallucinations, calculate the direction pointing from untruthful to truthful answers as the steering vector.
• Figure 4: True*Info scores split across subcategories on LLaMA-7B. The result reveals the significant performance enhancement of ACT across various subcategories in the TruthfulQA benchmark, compared to the baseline model.