Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Document-level Claim Extraction and Decontextualisation for Fact-Checking

Zhenyun Deng, Michael Schlichtkrull, Andreas Vlachos

TL;DR

The paper tackles the challenge of selecting check-worthy claims from multi-sentence documents by reframing document-level claim extraction as extractive summarization to identify central sentences, then enriching these sentences with document-wide context via a QA-driven decontextualisation process. It pairs this with a QA-to-context generation and a DeBERTa-based check-worthiness estimator to produce context-rich, unambiguous claims for verification. On the AVeriTeC and derived AVeriTeC-DCE datasets, the approach yields substantial gains over sentence-level baselines (e.g., $P@1=47.8$ vs $37.8$) and improves evidence retrieval (average $+1.08$ precision) through effective decontextualisation, with a $chrF$ of $26.4$ on gold decontextualised claims. These results demonstrate a practical pathway to scalable, document-aware fact-checking that better aligns with how audiences seek information and how evidence can be retrieved.”

Abstract

Selecting which claims to check is a time-consuming task for human fact-checkers, especially from documents consisting of multiple sentences and containing multiple claims. However, existing claim extraction approaches focus more on identifying and extracting claims from individual sentences, e.g., identifying whether a sentence contains a claim or the exact boundaries of the claim within a sentence. In this paper, we propose a method for document-level claim extraction for fact-checking, which aims to extract check-worthy claims from documents and decontextualise them so that they can be understood out of context. Specifically, we first recast claim extraction as extractive summarization in order to identify central sentences from documents, then rewrite them to include necessary context from the originating document through sentence decontextualisation. Evaluation with both automatic metrics and a fact-checking professional shows that our method is able to extract check-worthy claims from documents more accurately than previous work, while also improving evidence retrieval.

Document-level Claim Extraction and Decontextualisation for Fact-Checking

TL;DR

The paper tackles the challenge of selecting check-worthy claims from multi-sentence documents by reframing document-level claim extraction as extractive summarization to identify central sentences, then enriching these sentences with document-wide context via a QA-driven decontextualisation process. It pairs this with a QA-to-context generation and a DeBERTa-based check-worthiness estimator to produce context-rich, unambiguous claims for verification. On the AVeriTeC and derived AVeriTeC-DCE datasets, the approach yields substantial gains over sentence-level baselines (e.g., vs ) and improves evidence retrieval (average precision) through effective decontextualisation, with a of on gold decontextualised claims. These results demonstrate a practical pathway to scalable, document-aware fact-checking that better aligns with how audiences seek information and how evidence can be retrieved.”

Abstract

Selecting which claims to check is a time-consuming task for human fact-checkers, especially from documents consisting of multiple sentences and containing multiple claims. However, existing claim extraction approaches focus more on identifying and extracting claims from individual sentences, e.g., identifying whether a sentence contains a claim or the exact boundaries of the claim within a sentence. In this paper, we propose a method for document-level claim extraction for fact-checking, which aims to extract check-worthy claims from documents and decontextualise them so that they can be understood out of context. Specifically, we first recast claim extraction as extractive summarization in order to identify central sentences from documents, then rewrite them to include necessary context from the originating document through sentence decontextualisation. Evaluation with both automatic metrics and a fact-checking professional shows that our method is able to extract check-worthy claims from documents more accurately than previous work, while also improving evidence retrieval.
Paper Structure (28 sections, 7 equations, 4 figures, 9 tables)

This paper contains 28 sections, 7 equations, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Claim Extraction
  4. Decontextualisation
  5. Method
  6. Sentence Extraction
  7. Context Generation
  8. Question Generation.
  9. Question Answering.
  10. QA-to-Context Generation.
  11. Sentence Decontextualisation
  12. Check-Worthiness Estimation
  13. Dataset
  14. Experiments
  15. Sentence Extraction
  16. ...and 13 more sections

Figures (4)

  • Figure 1: An example of document-level claim extraction. Document is a piece of news from CNBC. Gold Claim is annotated by the fact-checking organization, Misbar. Sentences in orange denote check-worthy claims extracted by sentence-level CE (Claimbuster). Sentences in blue denote salient claims extracted by our document-level CE. The claim in green is a decontextualised claim derived from the 4th sentence obtained by our document-level CE.
  • Figure 2: An overview of our document-level claim extraction framework. Given an input document, we first use extractive summarization to rank all sentences and select summary sentences as central sentences. Then, we describe a QA-based framework to generate a specific high-quality context for important information units in the sentence. Next, we use a seq2seq generation model to decontextualise sentences by enriching them with their corresponding context. Finally, a claim check-worthiness classifier is used to select salient check-worthy claim sentences based on the score that reflects the degree to which sentences belong to the check-worthy claim.
  • Figure 3: Case studies of sentence decontextualisation solving linguistic problems, such as coreference resolution, global scoping and bridge anaphora.
  • Figure :