Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination

Zhuo Li, Yupeng Zhang, Pengyu Cheng, Jiajun Song, Mengyu Zhou, Hao Li, Shujie Hu, Yu Qin, Erchao Zhao, Xiaoxi Jiang, Guanjun Jiang

Abstract

Hallucination remains a critical bottleneck for large language models (LLMs), undermining their reliability in real-world applications, especially in Retrieval-Augmented Generation (RAG) systems. While existing hallucination detection methods employ LLM-as-a-judge to verify LLM outputs against retrieved evidence, they suffer from inherent confirmation bias, where the verifier inadvertently reproduces the errors of the original generation. To address this, we introduce Multi-Agent Reinforced Self-Check for Hallucination (MARCH), a framework that enforces rigorous factual alignment by leveraging deliberate information asymmetry. MARCH orchestrates a collaborative pipeline of three specialized agents: a Solver, a Proposer, and a Checker. The Solver generates an initial RAG response, which the Proposer decomposes into claim-level verifiable atomic propositions. Crucially, the Checker validates these propositions against retrieved evidence in isolation, deprived of the Solver's original output. This well-crafted information asymmetry scheme breaks the cycle of self-confirmation bias. By training this pipeline with multi-agent reinforcement learning (MARL), we enable the agents to co-evolve and optimize factual adherence. Extensive experiments across hallucination benchmarks demonstrate that MARCH substantially reduces hallucination rates. Notably, an 8B-parameter LLM equipped with MARCH achieves performance competitive with powerful closed-source models. MARCH paves a scalable path for factual self-improvement of LLMs through co-evolution. The code is at https://github.com/Qwen-Applications/MARCH.

MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination

Abstract

Hallucination remains a critical bottleneck for large language models (LLMs), undermining their reliability in real-world applications, especially in Retrieval-Augmented Generation (RAG) systems. While existing hallucination detection methods employ LLM-as-a-judge to verify LLM outputs against retrieved evidence, they suffer from inherent confirmation bias, where the verifier inadvertently reproduces the errors of the original generation. To address this, we introduce Multi-Agent Reinforced Self-Check for Hallucination (MARCH), a framework that enforces rigorous factual alignment by leveraging deliberate information asymmetry. MARCH orchestrates a collaborative pipeline of three specialized agents: a Solver, a Proposer, and a Checker. The Solver generates an initial RAG response, which the Proposer decomposes into claim-level verifiable atomic propositions. Crucially, the Checker validates these propositions against retrieved evidence in isolation, deprived of the Solver's original output. This well-crafted information asymmetry scheme breaks the cycle of self-confirmation bias. By training this pipeline with multi-agent reinforcement learning (MARL), we enable the agents to co-evolve and optimize factual adherence. Extensive experiments across hallucination benchmarks demonstrate that MARCH substantially reduces hallucination rates. Notably, an 8B-parameter LLM equipped with MARCH achieves performance competitive with powerful closed-source models. MARCH paves a scalable path for factual self-improvement of LLMs through co-evolution. The code is at https://github.com/Qwen-Applications/MARCH.

Paper Structure

This paper contains 64 sections, 8 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Formulation and Overview
  4. MARCH Modeling
  5. Overall Objective.
  6. MARCH Implementation
  7. Zero-Tolerance Reward (ZTR).
  8. Policy Optimization.
  9. Experiment
  10. Training Setup
  11. Training Datasets and Base Model.
  12. Training Details.
  13. Experiments on Hallucination
  14. Evaluation Benchmark.
  15. Evaluation Protocol and Metrics.
  16. ...and 49 more sections

Figures (4)

  • Figure 1: Overview of the MARCH framework. The Solver conducts retrieval-augmented generation base on the input query and related documents. The Proposer then decomposes this response into atomic propositions (the orange line in the response block) and formulates verifiable question-answer pairs. The Checker performs isolated verification by re-answering questions solely based on the retrieved evidence, without access to the Solver’s original output. This information-asymmetric pipeline, where the policy model plays all three roles, is optimized via Multi-Agent Reinforcement Learning (MARL) to achieve robust factual alignment.
  • Figure 2: Comparison of performance on the Facts Grounding. MARCH (highlighted in blue and purple) demonstrates competitive performance against a range of leading open and proprietary models.
  • Figure 3: Average number of proposed questions per step on the STEM training dataset. The dashed lines represent the MARCH framework without maintaining the question number, while the solid lines (w/ Constraint) indicate the results after incorporating instructional constraints to maintain informational density.
  • Figure 4: Visualization of training efficiency and dynamics. (a)-(d) illustrate the convergence of accuracy and rewards across general and STEM datasets. (e) Comparison of cumulative training time on the General dataset.