NAACL: Noise-AwAre Verbal Confidence Calibration for LLMs in RAG Systems

TL;DR

NAACL identifies a fundamental mismatch between verbal confidence and factual correctness in retrieval-augmented generation due to noisy retrieved contexts. It introduces Noise-AwAre Confidence CaLibration Rules (Conflict Independence, Noise Invariance, Parametric Fallback) and a self-bootstrapped training pipeline (NAACL) that synthesizes ~2K QA trajectories with explicit intermediate judgments. Through supervised fine-tuning on these trajectories, NAACL achieves significant calibration improvements (e.g., substantial reductions in $ECE$ and higher $AUROC$) while enhancing interpretability by grounding confidence in reasoning about passage utility. The framework remains general across noise levels and model backbones, offering a principled path toward epistemically reliable RAG systems, with limitations around model scale, synthetic noise, and long-context extension discussed.

Abstract

Accurately assessing model confidence is essential for deploying large language models (LLMs) in mission-critical factual domains. While retrieval-augmented generation (RAG) is widely adopted to improve grounding, confidence calibration in RAG settings remains poorly understood. We conduct a systematic study across four benchmarks, revealing that LLMs exhibit poor calibration performance due to noisy retrieved contexts. Specifically, contradictory or irrelevant evidence tends to inflate the model's false certainty, leading to severe overconfidence. To address this, we propose NAACL Rules (Noise-AwAre Confidence CaLibration Rules) to provide a principled foundation for resolving overconfidence under noise. We further design NAACL, a noise-aware calibration framework that synthesizes supervision from about 2K HotpotQA examples guided by these rules. By performing supervised fine-tuning (SFT) with this data, NAACL equips models with intrinsic noise awareness without relying on stronger teacher models. Empirical results show that NAACL yields substantial gains, improving ECE scores by 10.9% in-domain and 8.0% out-of-domain. By bridging the gap between retrieval noise and verbal calibration, NAACL paves the way for both accurate and epistemically reliable LLMs.

Figures (12)

• Figure 1: An illustrative example of model responses before and after NAACL. By explicitly training the model to assess passage- and group-level utility prior to answering, NAACL enables more reliable confidence expression under noisy retrieval, as reflected by consistently reduced ECE. The performance plots report results on NQ for Llama-3.1-8B-Instruct and DeepSeek-R1-Distill-Llama-8B, where SFT corresponds to the Label-only SFT setting in Table \ref{['table:main_results']}, and illustrate how NAACL promotes more transparent and grounded human–computer interaction in real-world scenarios.
• Figure 2: Calibration performance of Llama-3.1-8B-Instruct and DeepSeek-R1-Distill-Llama-8B on NQ and Bamboogle under controlled noise settings. The plots display ECE, AUROC, and Average Confidence across four retrieval settings: Gold-only, Gold+Irrelevant (Irr), Gold+Relevant (Rel), and Gold+Counterfactual (Cf). Results show that introducing noise, particularly counterfactual passages, substantially degrades calibration performance.
• Figure 3: Overview of the NAACL data pipeline with three stages: RAG Passage Construction, Training Response Generation, and Multi-stage Data Filtering. Specifically, In the Training Response Generation stage, the model takes a query $q$ and a set of retrieved passages $\mathcal{P}$ (where $k=3$) as input (denoted as Input: Q+3P). It then generates a reasoning trace containing passage-level and group-level judgments $J_p, J_g$ (denoted as P Type), followed by the predicted answer $\hat{a}$ (A) and the verbal confidence score $\hat{c}$ (C). Finally, the pipeline produces 2K high-quality trajectories used for fine-tuning.
• Figure 4: Reliability Diagram for HotpotQA: comparison of CoT prompt with base model (upper row) and SFT models (lower row). Each subplot displays accuracy v.s. confidence, with the diagonal dashed line representing perfect calibration.
• Figure 5: Case study setup illustrating a high-conflict retrieval scenario. The input consists of a query and three retrieved passages: the Ground Truth passage (Passage 1) is mixed with two Counterfactual passages (Passages 2 and 3) that support mutually exclusive incorrect answers ("Blargon-7" and "Omicron Persei 8"), testing the model's ability to handle contradictory evidence.