Retrieval-Augmented Generation with Conflicting Evidence

TL;DR

The paper tackles the challenge of conflicting evidence in retrieval-augmented generation by introducing RAMDocs, a realistic dataset that combines ambiguity, misinformation, and noise, and MADAM-RAG, a multi-agent debate framework with an aggregator for synthesizing reliable final answers. RAMDocs exposes retrieval imbalances and cross-source conflicts, while MADAM-RAG assigns each document to an independent agent that debates across rounds before an aggregator produces the final output, enabling both multiple valid answers and suppression of misinformation. Across FaithEval, AmbigDocs, and the challenging RamDocs, MADAM-RAG demonstrates consistent improvements over strong baselines, with substantial gains on AmbigDocs and FaithEval and notable resilience to increasing misinformation; RamDocs remains a difficult setting, highlighting the need for further advances. Overall, the work provides a unified approach to handling ambiguity, misinformation, and noise in RAG, introduces a meaningful benchmark (RamDocs) for real-world retrieval challenges, and points to future directions for more robust, trustworthy information-seeking systems.

Abstract

Large language model (LLM) agents are increasingly employing retrieval-augmented generation (RAG) to improve the factuality of their responses. However, in practice, these systems often need to handle ambiguous user queries and potentially conflicting information from multiple sources while also suppressing inaccurate information from noisy or irrelevant documents. Prior work has generally studied and addressed these challenges in isolation, considering only one aspect at a time, such as handling ambiguity or robustness to noise and misinformation. We instead consider multiple factors simultaneously, proposing (i) RAMDocs (Retrieval with Ambiguity and Misinformation in Documents), a new dataset that simulates complex and realistic scenarios for conflicting evidence for a user query, including ambiguity, misinformation, and noise; and (ii) MADAM-RAG, a multi-agent approach in which LLM agents debate over the merits of an answer over multiple rounds, allowing an aggregator to collate responses corresponding to disambiguated entities while discarding misinformation and noise, thereby handling diverse sources of conflict jointly. We demonstrate the effectiveness of MADAM-RAG using both closed and open-source models on AmbigDocs -- which requires presenting all valid answers for ambiguous queries -- improving over strong RAG baselines by up to 11.40% and on FaithEval -- which requires suppressing misinformation -- where we improve by up to 15.80% (absolute) with Llama3.3-70B-Instruct. Furthermore, we find that RAMDocs poses a challenge for existing RAG baselines (Llama3.3-70B-Instruct only obtains 32.60 exact match score). While MADAM-RAG begins to address these conflicting factors, our analysis indicates that a substantial gap remains especially when increasing the level of imbalance in supporting evidence and misinformation.

Figures (5)

• Figure 1: An example from our RamDocs dataset (left) illustrating multiple sources of conflict in retrieved documents. Conflict may occur because of ambiguity in the query (in this case, different referents for Michael Jordan), but also because of misinformation from incorrect documents and noise from irrelevant ones. Madam-RAG (right) addresses this through multi-agent debate, where each agent summarizes and represents the information in one document. Agents discuss their responses across multiple rounds, with the final answers being combined via an aggregator module that summarizes the discussion.
• Figure 2: Madam-RAG operates by assigning each document to a separate agent. Agents are responsible for summarizing and representing their documents, and engage in a multi-agent, multi-round debate with each other to filter out misinformation and noise and address ambiguity. An aggregator then summarizes the discussion into a single response.
• Figure 3: Ablation study on the importance of aggregator and multiple rounds of discussion.
• Figure 4: Performance of different methods with Llama3.3-70B-Instruct under setting (a) imbalance in supporting documents and (b) increasing the level of misinformation.
• Figure 5: Dataset statistics across eight dimensions. The first row shows document-level properties per example: total number of documents (avg: 5.53), number of documents supporting correct answers (avg: 3.84), incorrect answers (avg: 0.61), and noisy or irrelevant content (avg: 1.08). The second row presents answer-level properties: number of correct answers per example (avg: 2.20), wrong answers (avg: 0.86), and number of documents supporting each correct answer (avg: 1.77) and each wrong answer (avg: 0.73).