Tool-MAD: A Multi-Agent Debate Framework for Fact Verification with Diverse Tool Augmentation and Adaptive Retrieval

TL;DR

This work proposes Tool-MAD, a multi-agent debate framework that enhances factual verification by assigning each agent a distinct external tool, such as a search API or RAG module, and introduces three key innovations: a multi-agent debate framework where agents leverage heterogeneous external tools, encouraging diverse perspectives.

Abstract

Large Language Models (LLMs) suffer from hallucinations and factual inaccuracies, especially in complex reasoning and fact verification tasks. Multi-Agent Debate (MAD) systems aim to improve answer accuracy by enabling multiple LLM agents to engage in dialogue, promoting diverse reasoning and mutual verification. However, existing MAD frameworks primarily rely on internal knowledge or static documents, making them vulnerable to hallucinations. While MADKE introduces external evidence to mitigate this, its one-time retrieval mechanism limits adaptability to new arguments or emerging information during the debate. To address these limitations, We propose Tool-MAD, a multi-agent debate framework that enhances factual verification by assigning each agent a distinct external tool, such as a search API or RAG module. Tool-MAD introduces three key innovations: (1) a multi-agent debate framework where agents leverage heterogeneous external tools, encouraging diverse perspectives, (2) an adaptive query formulation mechanism that iteratively refines evidence retrieval based on the flow of the debate, and (3) the integration of Faithfulness and Answer Relevance scores into the final decision process, allowing the Judge agent to quantitatively assess the coherence and question alignment of each response and effectively detect hallucinations. Experimental results on four fact verification benchmarks demonstrate that Tool-MAD consistently outperforms state-of-the-art MAD frameworks, achieving up to 5.5% accuracy improvement. Furthermore, in medically specialized domains, Tool-MAD exhibits strong robustness and adaptability across various tool configurations and domain conditions, confirming its potential for broader real-world fact-checking applications.

Figures (11)

• Figure 1: Comparison of MAD liang-etal-2024-encouraging, MADKE wang2025learning and the proposed Tool-MAD: Unlike MAD and MADKE, which relies on a fixed document pool for retrieval, Tool-MAD dynamically retrieves external documents at each round through query formulation.
• Figure 2: Given a claim, two agents (RAG and Search) engage in multi-round debates, where $r$ denotes the current round and $T$ is the predefined round threshold. If no consensus is reached or the stability score falls below the threshold by round $T$, the Judge Agent issues a final verdict based on the debate history and the stability score.
• Figure 3: Empirical distributions of faithfulness and answer relevance for selecting stability-score thresholds. Answer relevance is concentrated above 0.8, while faithfulness shows a wider spread around 0.7–0.8. We therefore set thresholds to 0.7 (faithfulness) and 0.8 (answer relevance) to balance precision and efficiency
• Figure 4: Prompt of query selection
• Figure 5: Prompt of round debate