Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilities Are All You Need: A Probability-Only Approach to Uncertainty Estimation in Large Language Models

Manh Nguyen, Sunil Gupta, Hung Le

TL;DR

This paper tackles hallucinations in large language models by proposing PRO, a training-free uncertainty estimator that approximates the predictive entropy $H(Y|x)$ using the top-$K$ generation probabilities and an adaptive threshold $\alpha$ to filter low-probability outputs. The authors derive a lower bound $PRO(x) = -\log p^*_K - \sum_{i=1}^{K} p^*_i \log \frac{p^*_i}{p^*_K}$ and demonstrate that PRO improves uncertainty estimation across TriviaQA, SciQ, and Natural Questions datasets using multiple models, often outperforming more expensive baselines. The approach emphasizes simplicity and computational efficiency by relying solely on token probabilities, avoiding semantic embeddings or internal-state analyses. Empirical results show robust improvements in AUROC for uncertainty estimation, supporting PRO as a practical tool for enhancing the trustworthiness of LLM outputs. Limitations include dependence on token logits and lack of semantic consideration, suggesting avenues for integrating semantics and exploring alternative decoding strategies in future work.

Abstract

Large Language Models (LLMs) exhibit strong performance across various natural language processing (NLP) tasks but remain vulnerable to hallucinations, generating factually incorrect or misleading outputs. Uncertainty estimation, often using predictive entropy estimation, is key to addressing this issue. However, existing methods often require multiple samples or extra computation to assess semantic entropy. This paper proposes an efficient, training-free uncertainty estimation method that approximates predictive entropy using the responses' top-$K$ probabilities. Moreover, we employ an adaptive mechanism to determine $K$ to enhance flexibility and filter out low-confidence probabilities. Experimental results on three free-form question-answering datasets across several LLMs demonstrate that our method outperforms expensive state-of-the-art baselines, contributing to the broader goal of enhancing LLM trustworthiness.

Probabilities Are All You Need: A Probability-Only Approach to Uncertainty Estimation in Large Language Models

TL;DR

This paper tackles hallucinations in large language models by proposing PRO, a training-free uncertainty estimator that approximates the predictive entropy using the top- generation probabilities and an adaptive threshold to filter low-probability outputs. The authors derive a lower bound and demonstrate that PRO improves uncertainty estimation across TriviaQA, SciQ, and Natural Questions datasets using multiple models, often outperforming more expensive baselines. The approach emphasizes simplicity and computational efficiency by relying solely on token probabilities, avoiding semantic embeddings or internal-state analyses. Empirical results show robust improvements in AUROC for uncertainty estimation, supporting PRO as a practical tool for enhancing the trustworthiness of LLM outputs. Limitations include dependence on token logits and lack of semantic consideration, suggesting avenues for integrating semantics and exploring alternative decoding strategies in future work.

Abstract

Large Language Models (LLMs) exhibit strong performance across various natural language processing (NLP) tasks but remain vulnerable to hallucinations, generating factually incorrect or misleading outputs. Uncertainty estimation, often using predictive entropy estimation, is key to addressing this issue. However, existing methods often require multiple samples or extra computation to assess semantic entropy. This paper proposes an efficient, training-free uncertainty estimation method that approximates predictive entropy using the responses' top- probabilities. Moreover, we employ an adaptive mechanism to determine to enhance flexibility and filter out low-confidence probabilities. Experimental results on three free-form question-answering datasets across several LLMs demonstrate that our method outperforms expensive state-of-the-art baselines, contributing to the broader goal of enhancing LLM trustworthiness.

Paper Structure

This paper contains 17 sections, 1 theorem, 11 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Problem Statement
  5. Generation Probability
  6. Methodology
  7. Uncertainty Approximation
  8. On Top-$K$ Selection
  9. Experiments and Results
  10. Experimental Setup
  11. Benchmarking Results
  12. Model Analysis and Ablation Study
  13. Conclusion
  14. Appendix
  15. Proof of Proposition \ref{['prop:prop1']}
  16. ...and 2 more sections

Key Result

Proposition 1

Let $\mathbf{y}^* = (y^*_1, y^*_2, \dots, y^*_K)$ be the top $K$ generations of a LLM given prompt $x$. The predictive entropy approximation using the top $K$ probabilities satisfies the following inequality:

Figures (2)

  • Figure 1: AUC performance when adjusting $\alpha$ for various models and datasets.
  • Figure 2: Relationship between number of selected generations and $\alpha$ across different models and datasets.

Theorems & Definitions (2)

  • Proposition 1
  • proof