Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Truth-Aware Context Selection: Mitigating Hallucinations of Large Language Models Being Misled by Untruthful Contexts

Tian Yu, Shaolei Zhang, Yang Feng

TL;DR

Experimental results indicate that TACS can effectively filter untruthful context and significantly improve the overall quality of LLMs' responses when presented with misleading information.

Abstract

Although Large Language Models (LLMs) have demonstrated impressive text generation capabilities, they are easily misled by untruthful contexts provided by users or knowledge augmentation tools, leading to hallucinations. To alleviate LLMs from being misled by untruthful context and take advantage of knowledge augmentation, we propose Truth-Aware Context Selection (TACS), a lightweight method to adaptively recognize and mask untruthful context from the inputs. TACS begins by performing truth detection on the input context, leveraging the parameterized knowledge within the LLM. Subsequently, it constructs a corresponding attention mask based on the truthfulness of each position, selecting the truthful context and discarding the untruthful context. Additionally, we introduce a new evaluation metric, Disturbance Adaption Rate, to further study the LLMs' ability to accept truthful information and resist untruthful information. Experimental results indicate that TACS can effectively filter untruthful context and significantly improve the overall quality of LLMs' responses when presented with misleading information.

Truth-Aware Context Selection: Mitigating Hallucinations of Large Language Models Being Misled by Untruthful Contexts

TL;DR

Experimental results indicate that TACS can effectively filter untruthful context and significantly improve the overall quality of LLMs' responses when presented with misleading information.

Abstract

Although Large Language Models (LLMs) have demonstrated impressive text generation capabilities, they are easily misled by untruthful contexts provided by users or knowledge augmentation tools, leading to hallucinations. To alleviate LLMs from being misled by untruthful context and take advantage of knowledge augmentation, we propose Truth-Aware Context Selection (TACS), a lightweight method to adaptively recognize and mask untruthful context from the inputs. TACS begins by performing truth detection on the input context, leveraging the parameterized knowledge within the LLM. Subsequently, it constructs a corresponding attention mask based on the truthfulness of each position, selecting the truthful context and discarding the untruthful context. Additionally, we introduce a new evaluation metric, Disturbance Adaption Rate, to further study the LLMs' ability to accept truthful information and resist untruthful information. Experimental results indicate that TACS can effectively filter untruthful context and significantly improve the overall quality of LLMs' responses when presented with misleading information.
Paper Structure (31 sections, 9 equations, 9 figures, 8 tables, 1 algorithm)

This paper contains 31 sections, 9 equations, 9 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Construction of Classifiers for Truth Detection
  5. Generation with Truth-Aware Context Selection
  6. Disturbance Adaptation Rate
  7. Experiments
  8. Datasets
  9. Construction of Experimental Scenarios
  10. Experimental Setup
  11. Experimental Results
  12. Analysis
  13. Superiority of Truth-Aware Selection
  14. Generalization of TACS on More LLMs
  15. Variation of Truthfulness across Layers
  16. ...and 16 more sections

Figures (9)

  • Figure 1: Distributions of model-generated answers when a different type of information is provided. The figure illustrates that Llama 2-Chat 7B tends to select the answer supported by the given information, regardless of the truthfulness of the given information. The experiment is conducted on TruthfulQA. See Appendix \ref{['GMC']} for more details.
  • Figure 2: The diagram of our method. In (a), LLM is misled by untruthful information, resulting in hallucinations. On the contrary, (b) Truth-Aware Context Selection (TACS) uses classifiers to assess the truthfulness of the context. It masks out untruthful terms, thus reducing the risk of misleading LLMs into generating hallucinations.
  • Figure 3: Token-level truth detection Accuracy. Training rate represents the proportion of data used for training. The results are averaged over 5 runs.
  • Figure 4: The MC1 performance on TruthfulQA where double pieces of information are provided. The results are averaged over 3 runs.
  • Figure 5: Attention maps between answers (vertical axis) and information (horizontal axis). The deeper shades in color indicate higher attention scores from the answer to the information. Green tokens represent truthful information, while red tokens denote untruthful information.
  • ...and 4 more figures