Evidence-Driven Retrieval Augmented Response Generation for Online Misinformation

TL;DR

Evidence-driven retrieval augmented response generation (RARG) tackles online misinformation by grounding counter-responses in external scientific evidence. It combines a two-stage retrieval pipeline (BM25 followed by dense reranking) over a large academic corpus with RLHF-aligned generation to produce polite, factual rebuttals. The approach demonstrates strong evidence retrieval and generation quality on COVID-19 misinformation, outperforming baselines in both in-domain and cross-domain settings. This framework enables scalable, evidence-based debunking of misinformation with potential for broad-domain deployment.

Abstract

The proliferation of online misinformation has posed significant threats to public interest. While numerous online users actively participate in the combat against misinformation, many of such responses can be characterized by the lack of politeness and supporting facts. As a solution, text generation approaches are proposed to automatically produce counter-misinformation responses. Nevertheless, existing methods are often trained end-to-end without leveraging external knowledge, resulting in subpar text quality and excessively repetitive responses. In this paper, we propose retrieval augmented response generation for online misinformation (RARG), which collects supporting evidence from scientific sources and generates counter-misinformation responses based on the evidences. In particular, our RARG consists of two stages: (1) evidence collection, where we design a retrieval pipeline to retrieve and rerank evidence documents using a database comprising over 1M academic articles; (2) response generation, in which we align large language models (LLMs) to generate evidence-based responses via reinforcement learning from human feedback (RLHF). We propose a reward function to maximize the utilization of the retrieved evidence while maintaining the quality of the generated text, which yields polite and factual responses that clearly refutes misinformation. To demonstrate the effectiveness of our method, we study the case of COVID-19 and perform extensive experiments with both in- and cross-domain datasets, where RARG consistently outperforms baselines by generating high-quality counter-misinformation responses.

Figures (4)

• Figure 1: The proposed retrieval augmented response generation (RARG), which collects scientific evidence to generate counter-misinformation responses.
• Figure 2: The proposed two-stage retrieval pipeline in RARG (left) and its optimization (right).
• Figure 3: The optimization of $f_{\mathrm{gen}}$ in RARG. Upon input claim, the evidence documents are retrieved for response generation. Then, the reward model estimates the rewards and update the actor model with PPO-based RL.
• Figure 4: Hyperparameter sensitivity of $\lambda$ in retrieval.