Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Navigating Uncertainty: Optimizing API Dependency for Hallucination Reduction in Closed-Book Question Answering

Pierre Erbacher, Louis Falissar, Vincent Guigue, Laure Soulier

TL;DR

The paper tackles hallucination in closed-book QA by enabling LLMs to self-judge when to consult external sources. It introduces Hallucination Masking Mechanism (HalM), which replaces hallucinated outputs with an external-lookup token <search> while preserving correct direct answers, and it trains this behavior with parameter-efficient fine-tuning (LoRA) on CBQA data. Experiments on Natural Questions and TriviaQA show HalM reduces hallucinations and, with LoRA, often achieves a favorable balance between accuracy and external querying, outperforming perplexity-based baselines in several settings. This approach enables end-to-end, cost-aware grounding and has practical significance for deploying reliable, efficient API-augmented QA systems. The work also highlights the importance of tuning the trade-off parameter λ and demonstrates scalable methods for reducing external calls without sacrificing factual correctness.

Abstract

While Large Language Models (LLM) are able to accumulate and restore knowledge, they are still prone to hallucination. Especially when faced with factual questions, LLM cannot only rely on knowledge stored in parameters to guarantee truthful and correct answers. Augmenting these models with the ability to search on external information sources, such as the web, is a promising approach to ground knowledge to retrieve information. However, searching in a large collection of documents introduces additional computational/time costs. An optimal behavior would be to query external resources only when the LLM is not confident about answers. In this paper, we propose a new LLM able to self-estimate if it is able to answer directly or needs to request an external tool. We investigate a supervised approach by introducing a hallucination masking mechanism in which labels are generated using a close book question-answering task. In addition, we propose to leverage parameter-efficient fine-tuning techniques to train our model on a small amount of data. Our model directly provides answers for $78.2\%$ of the known queries and opts to search for $77.2\%$ of the unknown ones. This results in the API being utilized only $62\%$ of the time.

Navigating Uncertainty: Optimizing API Dependency for Hallucination Reduction in Closed-Book Question Answering

TL;DR

The paper tackles hallucination in closed-book QA by enabling LLMs to self-judge when to consult external sources. It introduces Hallucination Masking Mechanism (HalM), which replaces hallucinated outputs with an external-lookup token <search> while preserving correct direct answers, and it trains this behavior with parameter-efficient fine-tuning (LoRA) on CBQA data. Experiments on Natural Questions and TriviaQA show HalM reduces hallucinations and, with LoRA, often achieves a favorable balance between accuracy and external querying, outperforming perplexity-based baselines in several settings. This approach enables end-to-end, cost-aware grounding and has practical significance for deploying reliable, efficient API-augmented QA systems. The work also highlights the importance of tuning the trade-off parameter λ and demonstrates scalable methods for reducing external calls without sacrificing factual correctness.

Abstract

While Large Language Models (LLM) are able to accumulate and restore knowledge, they are still prone to hallucination. Especially when faced with factual questions, LLM cannot only rely on knowledge stored in parameters to guarantee truthful and correct answers. Augmenting these models with the ability to search on external information sources, such as the web, is a promising approach to ground knowledge to retrieve information. However, searching in a large collection of documents introduces additional computational/time costs. An optimal behavior would be to query external resources only when the LLM is not confident about answers. In this paper, we propose a new LLM able to self-estimate if it is able to answer directly or needs to request an external tool. We investigate a supervised approach by introducing a hallucination masking mechanism in which labels are generated using a close book question-answering task. In addition, we propose to leverage parameter-efficient fine-tuning techniques to train our model on a small amount of data. Our model directly provides answers for of the known queries and opts to search for of the unknown ones. This results in the API being utilized only of the time.
Paper Structure (14 sections, 3 equations, 2 figures, 3 tables)

This paper contains 14 sections, 3 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Learning when to search with LLMs
  3. Problem Formalization.
  4. Hallucination Masking Mechanism
  5. Evaluation protocol
  6. Datasets
  7. Metrics
  8. Model architectures and fine-tuning
  9. Baselines and model variants
  10. Results
  11. Conclusion
  12. Implementation Details
  13. Histogram Log-perplexity Hallucinations Rate
  14. Appendix

Figures (2)

  • Figure 1: Our objective is to teach an LLM to trade off between accessing an external tool and answering directly. The LLM should access an external knowledge base only when it is not self-confident about the answer, otherwise, it should generate the answer directly, minimizing the cost of accessing an external search engine.
  • Figure 2: Distribution of hallucinations between T5-XXL and T5-XXL-HMM (LoRA) by log-perplexity.