Reasoning-CV: Fine-tuning Powerful Reasoning LLMs for Knowledge-Assisted Claim Verification

TL;DR

The paper tackles knowledge-assisted claim verification by moving from a Decompose-Then-Verify framework to a CoT-Verify paradigm, arguing that end-to-end reasoning paths can yield higher fidelity judgments. It introduces Reasoning-CV, a two-stage fine-tuning approach that first distills reliable CoT paths from a high-capability model (Stage 1: SFT) and then iteratively improves reasoning quality and consistency via self-improvement direct preference optimization (Stage 2: DPO) with veracity-conditioned prompts. Experiments on FEVEROUS, HOVER, Healthver, and out-of-domain datasets show that Reasoning-CV enables an 8B open-source LLM to surpass strong baselines and even some black-box systems in both NEI and non-NEI settings, across in-domain and out-of-domain tasks. The work demonstrates the practical impact of high-quality, veracity-aligned CoT reasoning for robust claim verification and discusses limitations around knowledge retrieval in open-book settings and potential extensions to enumerable-answer tasks.

Abstract

Claim verification is essential in combating misinformation, and large language models (LLMs) have recently emerged in this area as powerful tools for assessing the veracity of claims using external knowledge. Existing LLM-based methods for claim verification typically adopt a Decompose-Then-Verify paradigm, which involves decomposing complex claims into several independent sub-claims and verifying each sub-claim separately. However, this paradigm often introduces errors during the claim decomposition process. To mitigate these errors, we propose to develop the Chain-of-Thought (CoT)-Verify paradigm, which leverages LLM reasoning methods to generate CoT-verification paths for the original complex claim without requiring decompositions into sub-claims and separate verification stages. The CoT-Verify paradigm allows us to propose a natural fine-tuning method called Reasoning-CV to enhance the verification capabilities in LLMs. Reasoning-CV includes a supervised fine-tuning (SFT) stage and a self-improvement direct preference optimization (DPO) stage. Utilizing only an 8B pre-trained LLM, Reasoning-CV demonstrates superior knowledge-assisted claim verification performances compared to existing Decompose-Then-Verify methods, as well as powerful black-box LLMs such as GPT-4o+CoT and o1-preview. Our code is available.

Figures (6)

• Figure 1: An example of claim decomposition using GPT-4o and prompts from min2023factscore. In this case, we get a redundant sub-claim (i.e., Sub-Claim 1 in Figure) that changes the original meaning.
• Figure 2: Existing LLM-based claim verification methods generally adopt a Decompose-Then-Verify paradigm (a). These methods first break a complex claim into several independent sub-claims, then leverage provided or retrieved knowledge to judge the veracity of each simple sub-claim. We propose to use a CoT-Verify paradigm (b), which aims at directly generating high-quality CoT-verification paths for the veracity of complex claims. It can reduce the number of LLM calls, eliminate the decomposition error, and achieve significantly better accuracy after the proposed Reasoning-CV fine-tuning.
• Figure 3: The proposed Reasoning-CV is a two-stage fine-tuning method for knowledge-assisted claim verification. Given a training dataset with Claim $C$, Knowledge $E$, and ground-truth veracity $V^*$, Reasoning-CV can obtain LLMs with high-quality CoT-verification paths. The first stage aims at distilling the GPT-4o generated reasoning path for ground-truth veracity $V^*$, and the second stage is designed to iteratively enhance the judgment and consistency of reasoning paths from the fine-tuned LLMs.
• Figure 4: A qualitative study for the possible error of reasoning LLMs after the first stage of Reasoning-CV fine-tuning. LLMs still face difficulties in judging the acceptance threshold (e.g., Error1) and in maintaining consistency for their long reasoning path (e.g., Error2).
• Figure 5: The evidence retrieval process in the open book setting of Reasoning-CV. We fine-tune another 8B LlaMA model for the claim decomposition and decontextualization tasks in min2023factscore. We retrieve the top paragraph from Google by the Serper API, taking each sub-claim as a query.