CRAVE: A Conflicting Reasoning Approach for Explainable Claim Verification Using LLMs

TL;DR

CRAVE addresses the challenge of verifying complex, multi-faceted claims by combining ambiguity-aware evidence retrieval, conflicting-perspective reasoning via LLMs, and a Small LM judge for final decision and explanations. The three-module architecture — Ambiguity Elimination enhanced Evidence Retrieval, Conflicting Perspective Reasoning with Preliminary Judgment, and SLM-based final judgment — achieves state-of-the-art accuracy on HOVER and FEVEROUS, while providing transparent rationales and linked evidence. Extensive ablations and analysis show the value of each component and the importance of explicit reasoning aspects in guiding the final verdict. The approach improves both verification performance and the quality of explanations, with practical impact for scalable, explainable misinformation detection and fact-checking workflows.

Abstract

The rapid spread of misinformation, driven by digital media and AI-generated content, has made automatic claim verification essential. Traditional methods, which depend on expert-annotated evidence, are labor-intensive and not scalable. Although recent automated systems have improved, they still struggle with complex claims that require nuanced reasoning. To address this, we propose CRAVE, a Conflicting Reasoning Approach for explainable claim VErification, that verify the complex claims based on the conflicting rationales reasoned by large language models (LLMs). Specifically, CRAVE introduces a three-module framework. Ambiguity Elimination enchanced Evidence Retrieval module performs ambiguity elimination and entity-based search to gather relevant evidence related to claim verification from external sources like Wikipedia. Conflicting Perspective Reasoning and Preliminary Judgment module with LLMs adopts LLMs to reason rationales with conflicting stances about claim verification from retrieved evidence across four dimensions, i.e., direct evidence, semantic relationships, linguistic patterns, and logical reasoning and make a preliminary judgment. Finally, Small Language Model (SLM) based Judge module is fine-tuned to make use of preliminary judgment from LLMs to assess the confidence of the conflicting rationales and make a final authenticity judgment. This methodology allows CRAVE to capture subtle inconsistencies in complex claims, improving both the accuracy and transparency of claim verification. Extensive experiments on two public claim verification datasets demonstrate that our CRAVE model achieves much better performance than state-of-the-art methods and exhibits a superior capacity for finding relevant evidence and explaining the model predictions. The code is provided at https://github.com/8zym/CRAVE.

Figures (3)

• Figure 1: The overview of CRAVE. It consists of three core module. a) Ambiguity Elimination enchanced Evidence Retrieval Module contains ambiguity elimination, entity retrieval and evidence retrieval and selection parts. b) Conflicting Perspective Reasoning and Preliminary Judgment module with LLMs that reason why the evidence supports or refutes the claim from four reasoning aspects and make a preliminary Judgment using LLMs. c) Judge module adopts SLM to make the final decision.
• Figure 2: Simple example for the ambiguity elimination and entity retrieval in Ambiguity Elimination enchanced Evidence Retrieval Module. After these procedures, CRAVE will find evidence based on the entities.
• Figure 3: Automatic GPT-4 evaluation of the explanation quality over different reasoning aspects. (a) Evaluation over HOVER dataset. (b) Evluation over FEVEROUS dataset