Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mitigating Hallucination on Hallucination in RAG via Ensemble Voting

Zequn Xie, Zhengyang Sun

Abstract

Retrieval-Augmented Generation (RAG) aims to reduce hallucinations in Large Language Models (LLMs) by integrating external knowledge. However, RAG introduces a critical challenge: hallucination on hallucination," where flawed retrieval results mislead the generation model, leading to compounded hallucinations. To address this issue, we propose VOTE-RAG, a novel, training-free framework with a two-stage structure and efficient, parallelizable voting mechanisms. VOTE-RAG includes: (1) Retrieval Voting, where multiple agents generate diverse queries in parallel and aggregate all retrieved documents; (2) Response Voting, where multiple agents independently generate answers based on the aggregated documents, with the final output determined by majority vote. We conduct comparative experiments on six benchmark datasets. Our results show that VOTE-RAG achieves performance comparable to or surpassing more complex frameworks. Additionally, VOTE-RAG features a simpler architecture, is fully parallelizable, and avoids the problem drift" risk. Our work demonstrates that simple, reliable ensemble voting is a superior and more efficient method for mitigating RAG hallucinations.

Mitigating Hallucination on Hallucination in RAG via Ensemble Voting

Abstract

Retrieval-Augmented Generation (RAG) aims to reduce hallucinations in Large Language Models (LLMs) by integrating external knowledge. However, RAG introduces a critical challenge: hallucination on hallucination," where flawed retrieval results mislead the generation model, leading to compounded hallucinations. To address this issue, we propose VOTE-RAG, a novel, training-free framework with a two-stage structure and efficient, parallelizable voting mechanisms. VOTE-RAG includes: (1) Retrieval Voting, where multiple agents generate diverse queries in parallel and aggregate all retrieved documents; (2) Response Voting, where multiple agents independently generate answers based on the aggregated documents, with the final output determined by majority vote. We conduct comparative experiments on six benchmark datasets. Our results show that VOTE-RAG achieves performance comparable to or surpassing more complex frameworks. Additionally, VOTE-RAG features a simpler architecture, is fully parallelizable, and avoids the problem drift" risk. Our work demonstrates that simple, reliable ensemble voting is a superior and more efficient method for mitigating RAG hallucinations.

Paper Structure

This paper contains 19 sections, 15 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Limitations of Standard RAG
  4. Multi-Agent Ensembles and Voting
  5. Robustness and Disambiguation in Complex Retrieval
  6. The VOTE-RAG Framework
  7. THE VOTE-RAG FRAMEWORK
  8. Retrieval Voting
  9. Response Voting
  10. intuition and error bounds
  11. Overview
  12. Experimental Setup
  13. Datasets
  14. Baselines
  15. Implementation Details
  16. ...and 4 more sections

Figures (1)

  • Figure 1: An overview of our VOTE-RAG framework. It leverages parallel ensemble voting to enhance retrieval breadth and generation robustness against "Hallucination on Hallucination."