Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations

TL;DR

The paper investigates abstention based on model uncertainty to improve the reliability of large language models in QA tasks. It distinguishes statistical uncertainty (from token distributions) and In-Dialogue Uncertainty (hedging in responses) and evaluates them on correctness, hallucination avoidance, and safety across LLama2 variants with and without RLHF. Results show that selecting the right uncertainty signal—statistical for correctness and safety, InDU for unanswerable questions—substantially boosts accuracy (2–8%), halves hallucinations, and raises safe responses by up to 99% with minimal overhead. This work highlights uncertainty as a practical lever for deploying LLMs in high-stakes settings, guiding abstention policies that improve reliability without heavy computational costs.

Abstract

A major barrier towards the practical deployment of large language models (LLMs) is their lack of reliability. Three situations where this is particularly apparent are correctness, hallucinations when given unanswerable questions, and safety. In all three cases, models should ideally abstain from responding, much like humans, whose ability to understand uncertainty makes us refrain from answering questions we don't know. Inspired by analogous approaches in classification, this study explores the feasibility and efficacy of abstaining while uncertain in the context of LLMs within the domain of question-answering. We investigate two kinds of uncertainties, statistical uncertainty metrics and a distinct verbalized measure, termed as In-Dialogue Uncertainty (InDU). Using these uncertainty measures combined with models with and without Reinforcement Learning with Human Feedback (RLHF), we show that in all three situations, abstention based on the right kind of uncertainty measure can boost the reliability of LLMs. By sacrificing only a few highly uncertain samples we can improve correctness by 2% to 8%, avoid 50% hallucinations via correctly identifying unanswerable questions and increase safety by 70% up to 99% with almost no additional computational overhead.

Figures (12)

• Figure 1: Abstention based on the right form of uncertainty improves correctness, hallucinations and safety in LLMs.
• Figure 2: RLHF finetuning leads to higher confidence and less diversity in responses. These figures present a comparison between Llama2-70b pretrained and RLHF finetuned across various datasets. RLHF finetuned models exhibit (a) higher predictive confidence (lower mean predictive entropies across samples) and (b) are less diverse (lower number of semantic sets across samples) than pretrained models.
• Figure 3: Uncertainty-based Abstention leads to improvements in correctness, hallucinations and safety when using the right uncertainty measures. This figure shows Accuracy-Rejection Curves (ARCs) for correctness and safety and a Receiver Operating characteristics (ROC) curve with false abstention referring to the false positive rate for hallucination settings. Across all three scenarios, adopting an abstention approach for uncertain responses enhances accuracy w.r.t. correctness, improves the detection of unanswerable questions thereby reducing hallucinations, and boosts the safe response rate.
• Figure 4: Accuracy-Rejection Curves (ARCs) for TriviaQA and SciQA: Accuracy is defined as the ratio of correct samples to all samples in the remaining dataset. Rejection rate denotes the proportion of progressively rejected samples based on the uncertainty measure.
• Figure 5: Accuracy-Rejection Curves (ARCs) for AttaQ and AutoDAN: Accuracy is defined as the ratio of safe samples to all samples in the remaining dataset. Rejection rate denotes the proportion of progressively rejected samples based on the uncertainty measure.