Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Courtroom-Style Multi-Agent Debate with Progressive RAG and Role-Switching for Controversial Claim Verification

Masnun Nuha Chowdhury, Nusrat Jahan Beg, Umme Hunny Khan, Syed Rifat Raiyan, Md Kamrul Hasan, Hasan Mahmud

Abstract

Large language models (LLMs) remain unreliable for high-stakes claim verification due to hallucinations and shallow reasoning. While retrieval-augmented generation (RAG) and multi-agent debate (MAD) address this, they are limited by one-pass retrieval and unstructured debate dynamics. We propose a courtroom-style multi-agent framework, PROClaim, that reformulates verification as a structured, adversarial deliberation. Our approach integrates specialized roles (e.g., Plaintiff, Defense, Judge) with Progressive RAG (P-RAG) to dynamically expand and refine the evidence pool during the debate. Furthermore, we employ evidence negotiation, self-reflection, and heterogeneous multi-judge aggregation to enforce calibration, robustness, and diversity. In zero-shot evaluations on the Check-COVID benchmark, PROClaim achieves 81.7% accuracy, outperforming standard multi-agent debate by 10.0 percentage points, with P-RAG driving the primary performance gains (+7.5 pp). We ultimately demonstrate that structural deliberation and model heterogeneity effectively mitigate systematic biases, providing a robust foundation for reliable claim verification. Our code and data are publicly available at https://github.com/mnc13/PROClaim.

Courtroom-Style Multi-Agent Debate with Progressive RAG and Role-Switching for Controversial Claim Verification

Abstract

Large language models (LLMs) remain unreliable for high-stakes claim verification due to hallucinations and shallow reasoning. While retrieval-augmented generation (RAG) and multi-agent debate (MAD) address this, they are limited by one-pass retrieval and unstructured debate dynamics. We propose a courtroom-style multi-agent framework, PROClaim, that reformulates verification as a structured, adversarial deliberation. Our approach integrates specialized roles (e.g., Plaintiff, Defense, Judge) with Progressive RAG (P-RAG) to dynamically expand and refine the evidence pool during the debate. Furthermore, we employ evidence negotiation, self-reflection, and heterogeneous multi-judge aggregation to enforce calibration, robustness, and diversity. In zero-shot evaluations on the Check-COVID benchmark, PROClaim achieves 81.7% accuracy, outperforming standard multi-agent debate by 10.0 percentage points, with P-RAG driving the primary performance gains (+7.5 pp). We ultimately demonstrate that structural deliberation and model heterogeneity effectively mitigate systematic biases, providing a robust foundation for reliable claim verification. Our code and data are publicly available at https://github.com/mnc13/PROClaim.

Paper Structure

This paper contains 57 sections, 14 equations, 5 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Argument Mining
  4. Initial Evidence Retrieval
  5. Evidence Negotiation and Judicial Arbitration
  6. Progressive Retrieval-Augmented Generation (P-RAG)
  7. Multi-Agent Debate (MAD) Orchestration
  8. Self-Reflection and Critic Evaluation
  9. Role-Switching Consistency Test
  10. Judicial Panel Evaluation and Final Verdict
  11. Experimental Setup
  12. Results and Discussion
  13. Main Pipeline Performance
  14. Debate Dynamics and Adaptive Stopping
  15. Comparison with Baselines
  16. ...and 42 more sections

Figures (5)

  • Figure 1: Overview of the pipeline
  • Figure 2: Termination distribution and convergence speed across 360 debate instances.
  • Figure 3: P-RAG evidence novelty across debate rounds
  • Figure 4: Reflection score trajectories across plateau, judicial, and critic resolution patterns
  • Figure 5: Cost–accuracy Pareto front across system configurations. PROClaim (Full) lies on the efficient frontier; the dashed line traces Pareto-optimal points. w/o P-RAG is strictly dominated: it saves fewer tokens than w/o Role-Switch while incurring a larger accuracy penalty.