Debating Truth: Debate-driven Claim Verification with Multiple Large Language Model Agents

TL;DR

DebateCV introduces a debate-driven claim verification framework with two opposing LLM Debaters and a Moderator, addressing weaknesses of single-agent verification on complex evidence. To train the Moderator for multi-round adjudication, the authors propose Debate-SFT, which creates SynDeC, a synthetic debate dataset built from zero-shot DebateCV outputs and corrected by a ground-truth-guided Corrector. Extensive experiments on the AVeriTeC benchmark show DebateCV achieves higher accuracy than state-of-the-art baselines across evidence conditions and yields superior justification quality according to human evaluators. The work demonstrates that adversarial, cross-agent debate improves both verdict accuracy and interpretability, offering a scalable approach to reducing misinformation and enhancing digital literacy.

Abstract

Claim verification is essential for digital literacy, yet state-of-the-art single-agent methods often struggle with complex claims that require nuanced analysis of multifaceted online evidence. Inspired by real-world human fact-checking practices, we propose \textbf{DebateCV}, the first debate-driven claim verification framework powered by multiple LLM agents. In DebateCV, two \textit{Debaters} argue opposing stances over multiple rounds to surface subtle errors in single-agent assessments. A decisive \textit{Moderator} is then required to weigh the evidential strength of conflicting arguments to deliver an accurate verdict. Yet zero-shot agents struggle to adjudicate multi-round debates for verifying complex claims, often defaulting to neutral judgements, and no datasets exist for training agents for this role. To bridge this gap, we propose \textbf{Debate-SFT}, a post-training framework that leverages synthetic data to enhance agents' ability to effectively adjudicate debates for claim verification. Results show that our methods surpass state-of-the-art non-debate approaches in both accuracy (across various evidence conditions) and justification quality, which strengthens societal resilience against misinformation and contributes to a more trustworthy online information ecosystem.

Figures (3)

• Figure 1: An overview of DebateCV and Debate-SFT.
• Figure 2: Claim verification performance across debated rounds.
• Figure 3: Human evaluation results on the quality of the generated debates and justifications.