Systematic Evaluation of Uncertainty Estimation Methods in Large Language Models

TL;DR

This work systematically benchmarks four uncertainty estimation strategies for large language model outputs across four QA tasks, using a state-of-the-art open-source LLM. It analyzes calibration via ECE and discrimination via AUROC, showing that verbalized confidence often overestimates certainty while multi-sample aggregation and the CoCoA fusion improve both calibration and discrimination. The study provides a practical toolkit, including a unified implementation, compact calibration scores, and multi-dataset guidance, highlighting when each method is most effective. The results support deploying CoCoA for calibrated reliability, MSP for ranking when knowledge uncertainty dominates, and TOP-K VCE as a compute-efficient alternative, with clear task-dependent recommendations and a call for future work on longer outputs and larger models.

Abstract

Large language models (LLMs) produce outputs with varying levels of uncertainty, and, just as often, varying levels of correctness; making their practical reliability far from guaranteed. To quantify this uncertainty, we systematically evaluate four approaches for confidence estimation in LLM outputs: VCE, MSP, Sample Consistency, and CoCoA (Vashurin et al., 2025). For the evaluation of the approaches, we conduct experiments on four question-answering tasks using a state-of-the-art open-source LLM. Our results show that each uncertainty metric captures a different facet of model confidence and that the hybrid CoCoA approach yields the best reliability overall, improving both calibration and discrimination of correct answers. We discuss the trade-offs of each method and provide recommendations for selecting uncertainty measures in LLM applications.

Figures (9)

• Figure 1: VCE calibration on SQuAD 2.0 across decoding methods. Reliability plots (empirical accuracy vs. mean confidence) for single-sample, multi-sample with single decoding (TOP-K), and multi-sample with multi-decoding (SEP). Bubble area is proportional to bin count; the dashed line denotes perfect calibration $y{=}x$. Multi-decoding tightens high-confidence bins, reduces ECE, and improves AUROC while keeping accuracy comparable.
• Figure 2: Temperature effects on ranking quality. AUROC vs. decoding temperature for multi-sample VCE under SEP (solid) and TOP-K (dashed). Optima differ by dataset, with most gains occurring at mid-range $T$ (0.6–0.8).
• Figure 3: Calibration of CoCoA on TriviaQA. Bin-wise accuracy increases monotonically with confidence and aligns closely with the $y{=}x$ diagonal at high confidence, indicating well-calibrated scores in the decision-relevant range (bin sizes annotated).
• Figure A.1: Single-sample VCE temperature sweep. Optima are task-specific; no single $T$ dominates across datasets.
• Figure A.2: Multi-sample VCE temperature sweep. Optima are task-specific; no single $T$ dominates across datasets; SEP and TOP-K are broadly comparable.