Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ConfRAG: Confidence-Guided Retrieval-Augmenting Generation

Yin Huang, Yifan Ethan Xu, Kai Sun, Vera Yan, Alicia Sun, Haidar Khan, Jimmy Nguyen, Jingxiang Chen, Mohammad Kachuee, Zhaojiang Lin, Yue Liu, Aaron Colak, Anuj Kumar, Wen-tau Yih, Xin Luna Dong

TL;DR

ConfRAG addresses factual hallucinations and retrieval costs by training a confidence-calibrated QA module (ConfQA) that answers only when certain, and defers to RAG when unsure. This unsureness signal then triggers a retrieval Augmented pipeline (ConfRAG), achieving high accuracy in ideal conditions while substantially reducing unnecessary external retrievals. Across seven benchmarks, ConfQA reduces hallucinations to below 5% and ConfRAG delivers strong triggering performance, with real-RAG deployments yielding competitive accuracy and meaningful latency savings. The work formalizes a practical framework for integrating calibrated internal knowledge with external sources to improve factuality and efficiency in real-world AI systems.

Abstract

Can Large Language Models (LLMs) be trained to avoid hallucinating factual statements, and can Retrieval-Augmented Generation (RAG) be triggered only when necessary to reduce retrieval and computation costs? In this work, we address both challenges simultaneously. We introduce ConfQA, a fine-tuning strategy that reduces hallucination rates from 20-40% to below 5% across multiple factuality benchmarks. The approach is simple: when the model answers correctly, it is trained to output the answer; otherwise, it is trained to respond with "I am unsure". Two design choices make this training effective: (1) a dampening prompt ("answer only if you are confident") that explicitly discourages overconfident hallucinations, and (2) training data drawn from atomic factual statements (e.g., knowledge graph attribute values), which calibrates model confidence and yields robust generalization across domains and question types. Building on ConfQA, we propose ConfRAG, a triggering strategy that invokes RAG only when the model responses with unsure. This framework achieves accuracy above 95% in ideal case while reducing unnecessary external retrievals by over 30%.

ConfRAG: Confidence-Guided Retrieval-Augmenting Generation

TL;DR

ConfRAG addresses factual hallucinations and retrieval costs by training a confidence-calibrated QA module (ConfQA) that answers only when certain, and defers to RAG when unsure. This unsureness signal then triggers a retrieval Augmented pipeline (ConfRAG), achieving high accuracy in ideal conditions while substantially reducing unnecessary external retrievals. Across seven benchmarks, ConfQA reduces hallucinations to below 5% and ConfRAG delivers strong triggering performance, with real-RAG deployments yielding competitive accuracy and meaningful latency savings. The work formalizes a practical framework for integrating calibrated internal knowledge with external sources to improve factuality and efficiency in real-world AI systems.

Abstract

Can Large Language Models (LLMs) be trained to avoid hallucinating factual statements, and can Retrieval-Augmented Generation (RAG) be triggered only when necessary to reduce retrieval and computation costs? In this work, we address both challenges simultaneously. We introduce ConfQA, a fine-tuning strategy that reduces hallucination rates from 20-40% to below 5% across multiple factuality benchmarks. The approach is simple: when the model answers correctly, it is trained to output the answer; otherwise, it is trained to respond with "I am unsure". Two design choices make this training effective: (1) a dampening prompt ("answer only if you are confident") that explicitly discourages overconfident hallucinations, and (2) training data drawn from atomic factual statements (e.g., knowledge graph attribute values), which calibrates model confidence and yields robust generalization across domains and question types. Building on ConfQA, we propose ConfRAG, a triggering strategy that invokes RAG only when the model responses with unsure. This framework achieves accuracy above 95% in ideal case while reducing unnecessary external retrievals by over 30%.

Paper Structure

This paper contains 34 sections, 3 equations, 6 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. RAG with external knowledge:
  4. LLM hallucination suppression:
  5. Methodology
  6. Problem Definition
  7. ConfQA: When to Say Unsure?
  8. ConfRAG: When to Trigger RAG?
  9. Experiment Setup
  10. Benchmarks and Metrics
  11. Data sets:
  12. Metrics and evaluation:
  13. LLM Models and implementations:
  14. Analysis of LLM's Self-Reported Confidence (RQ1)
  15. Experimental Results
  16. ...and 19 more sections

Figures (6)

  • Figure 1: Factuality improvement of ConfRAG and fine-tuned model ConfQA.
  • Figure 2: ConfRAG invocation architecture.
  • Figure 3: LLMs' self-reported confidences is correlated with QA accuracy, but often over-confident.
  • Figure 4: ConfQA suppresses more (gives more missing answers) on less popular entities (missing rate for Tail is larger than Torso, whose missing rate is larger than Head).
  • Figure 5: Correlation between LLM's self-reported confidences and average accuracies on the CRAG dataset and the Head-to-Tail dataset, categorized by question types.
  • ...and 1 more figures