Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation

TL;DR

This work addresses the problem of unreliable uncertainty estimation in natural language generation due to mismatches between sampled outputs and label sources. It introduces Label-Confidence-Aware (LCA) uncertainty estimation, which uses KL-divergence between the sampling distribution and label sources, anchored by Gibbs probability, to produce a more robust uncertainty measure. Across multiple models and datasets, LCA consistently improves AUROC for trust decisions and highlights the importance of accounting for label confidence alongside sampling entropy. The approach achieves higher reliability with minimal overhead, offering a practical path to safer and more trustworthy NLG systems.

Abstract

Large Language Models (LLMs) display formidable capabilities in generative tasks but also pose potential risks due to their tendency to generate hallucinatory responses. Uncertainty Quantification (UQ), the evaluation of model output reliability, is crucial for ensuring the safety and robustness of AI systems. Recent studies have concentrated on model uncertainty by analyzing the relationship between output entropy under various sampling conditions and the corresponding labels. However, these methods primarily focus on measuring model entropy with precision to capture response characteristics, often neglecting the uncertainties associated with greedy decoding results-the sources of model labels, which can lead to biased classification outcomes. In this paper, we explore the biases introduced by greedy decoding and propose a label-confidence-aware (LCA) uncertainty estimation based on Kullback-Leibler (KL) divergence bridging between samples and label source, thus enhancing the reliability and stability of uncertainty assessments. Our empirical evaluations across a range of popular LLMs and NLP datasets reveal that different label sources can indeed affect classification, and that our approach can effectively capture differences in sampling results and label sources, demonstrating more effective uncertainty estimation.

Figures (3)

• Figure 1: Ignoring the probability information of the label answer in Free-form may lead to incorrect uncertain classification. We term it as label confidence unawareness, and integrate the omitted information into our method.
• Figure 2: Percentage of Falcon-7B and Mistral-7B w. & w/o label answers in sample on CoQA and TriviaQA.
• Figure 3: Ablation results. (a):Num of generation ablation. As number rises, AUROCs increase and then levels off.(b)ROUGE-L threshold ablation. As the higher threshold is, a stricter critirion it is and the better result we get. (c)TriviaQA temperature ablation on Llama2-7B. As the temperature rises, AUROCs first increase and then decrease