Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CORRECT: Context- and Reference-Augmented Reasoning and Prompting for Fact-Checking

Delvin Ce Zhang, Dongwon Lee

TL;DR

The paper addresses the challenge of fact-checking claims when evidence is not self-contained and may require surrounding context and references. It introduces CORRECT, which combines a three-layer evidence graph (evidence, context, reference) with intra- and cross-layer reasoning and an evidence-conditioned prompting scheme that generates claim-specific prompt embeddings for verdict prediction. Across four diverse datasets, CORRECT demonstrates strong improvements over a broad set of baselines under both fully supervised and few-shot settings, and its effectiveness persists with both gold and retrieved evidence. The work offers a practical approach to integrating auxiliary information in factual verification and points to future extensions into multi-modal evidence graphs for broader applicability.

Abstract

Fact-checking the truthfulness of claims usually requires reasoning over multiple evidence sentences. Oftentimes, evidence sentences may not be always self-contained, and may require additional contexts and references from elsewhere to understand coreferential expressions, acronyms, and the scope of a reported finding. For example, evidence sentences from an academic paper may need contextual sentences in the paper and descriptions in its cited papers to determine the scope of a research discovery. However, most fact-checking models mainly focus on the reasoning within evidence sentences, and ignore the auxiliary contexts and references. To address this problem, we propose a novel method, Context- and Reference-augmented Reasoning and Prompting. For evidence reasoning, we construct a three-layer evidence graph with evidence, context, and reference layers. We design intra- and cross-layer reasoning to integrate three graph layers into a unified evidence embedding. For verdict prediction, we design evidence-conditioned prompt encoder, which produces unique prompt embeddings for each claim. These evidence-conditioned prompt embeddings and claims are unified for fact-checking. Experiments verify the strength of our model.

CORRECT: Context- and Reference-Augmented Reasoning and Prompting for Fact-Checking

TL;DR

The paper addresses the challenge of fact-checking claims when evidence is not self-contained and may require surrounding context and references. It introduces CORRECT, which combines a three-layer evidence graph (evidence, context, reference) with intra- and cross-layer reasoning and an evidence-conditioned prompting scheme that generates claim-specific prompt embeddings for verdict prediction. Across four diverse datasets, CORRECT demonstrates strong improvements over a broad set of baselines under both fully supervised and few-shot settings, and its effectiveness persists with both gold and retrieved evidence. The work offers a practical approach to integrating auxiliary information in factual verification and points to future extensions into multi-modal evidence graphs for broader applicability.

Abstract

Fact-checking the truthfulness of claims usually requires reasoning over multiple evidence sentences. Oftentimes, evidence sentences may not be always self-contained, and may require additional contexts and references from elsewhere to understand coreferential expressions, acronyms, and the scope of a reported finding. For example, evidence sentences from an academic paper may need contextual sentences in the paper and descriptions in its cited papers to determine the scope of a research discovery. However, most fact-checking models mainly focus on the reasoning within evidence sentences, and ignore the auxiliary contexts and references. To address this problem, we propose a novel method, Context- and Reference-augmented Reasoning and Prompting. For evidence reasoning, we construct a three-layer evidence graph with evidence, context, and reference layers. We design intra- and cross-layer reasoning to integrate three graph layers into a unified evidence embedding. For verdict prediction, we design evidence-conditioned prompt encoder, which produces unique prompt embeddings for each claim. These evidence-conditioned prompt embeddings and claims are unified for fact-checking. Experiments verify the strength of our model.

Paper Structure

This paper contains 13 sections, 13 equations, 5 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Model Architecture
  4. Problem Formulation
  5. Three-layer Evidence Graph Reasoning
  6. Evidence-conditioned Prompting
  7. Experiments
  8. Empirical Evaluation
  9. Model Analysis
  10. Conclusion
  11. Pseudo-code of Training Process
  12. Dataset Preprocessing Details
  13. Experiment Environment

Figures (5)

  • Figure 1: Illustration of (a) context-dependent and (b) reference-dependent evidence from BearFact dataset.
  • Figure 2: Model architecture. (a) A three-layer graph for a claim. (b) Intra- and cross-layer reasoning. (c) A nested architecture with language model and graph reasoning for evidence encoding. (d) Evidence-conditioned prompting.
  • Figure 3: Few-shot veracity prediction with different number of shots.
  • Figure 4: Model analysis on Check-COVID and FEVEROUS-S.
  • Figure 5: Case study on BearFact dataset.