Grounding Fallacies Misrepresenting Scientific Publications in Evidence

TL;DR

MissciPlus extends Missci by grounding misrepresented health claims in real-world publication passages and formalizing three reconstruction subtasks: finding the kernel of truth, locating undermining passages, and reconstructing fallacious arguments. The study benchmarks lexical/semantic passage retrieval, various AFC models, and LLMs, revealing that current methods struggle to refute misinformation when evidence is grounded in real passages, and that LLMs can be misled by such evidence. It highlights the challenges of grounding implicit fallacies and the need for robust evidence-grounding in AFC, while addressing dual-use and ethical considerations. The work provides a realistic test-bed for detection and debunking, and outlines future directions including synthetic data generation and multi-document grounding to improve resilience against health misinformation.

Abstract

Health-related misinformation claims often falsely cite a credible biomedical publication as evidence. These publications only superficially seem to support the false claim, when logical fallacies are applied. In this work, we aim to detect and to highlight such fallacies, which requires assessing the exact content of the misrepresented publications. To achieve this, we introduce MissciPlus, an extension of the fallacy detection dataset Missci. MissciPlus extends Missci by grounding the applied fallacies in real-world passages from misrepresented studies. This creates a realistic test-bed for detecting and verbalizing fallacies under real-world input conditions, and enables new and realistic passage-retrieval tasks. MissciPlus is the first logical fallacy dataset which pairs the real-world misrepresented evidence with incorrect claims, identical to the input to evidence-based fact-checking models. With MissciPlus, we i) benchmark retrieval models in identifying passages that support claims only with fallacious reasoning, ii) evaluate how well LLMs verbalize fallacious reasoning based on misrepresented scientific passages, and iii) assess the effectiveness of fact-checking models in refuting claims that misrepresent biomedical research. Our findings show that current fact-checking models struggle to use misrepresented scientific passages to refute misinformation. Moreover, these passages can mislead LLMs into accepting false claims as true.

Figures (24)

• Figure 1: We link the paraphrased context from Missci to real-world passages. The LLM must (i) find relevant passages from the original study and (ii) generate a fallacious premise to (falsely) support the claim.
• Figure 2: A real-world passage vincent2005chloroquine communicates the paraphrased content $s_1$ from Missci that the study used cell cultures for their experiments.
• Figure 3: Recall of undermining passages per fallacy class (and accurate premise) over the top $k$ ranked passages. We only list fallacies with $\geq 20$ occurrences.
• Figure 4: Examples for the (D)efinition, (L)ogical form and (E)xample for the Fallacy of Composition, used as supplementary fallacy information in the prompts.
• Figure 5: AFC predictions over passages linked to the accurate premise (top left), to reasoning gaps (top right), to both (bottom left) or none (bottom right).