ClaimPT: A Portuguese Dataset of Annotated Claims in News Articles

TL;DR

The paper introduces ClaimPT, a European Portuguese dataset of 1,308 news articles annotated at the claim level to support automated fact-checking in low-resource languages. It details a multi-layer annotation scheme that links article metadata, claim spans, target objects, claimers, time, stance, and identity/attribute links, with two annotators and curator validation, using INCEpTION and a public GitHub resource. Baseline experiments compare encoder-based models (notably BERTimbau with sentence segmentation) and generative Gemini models, showing span-based, encoder approaches outperform generative baselines, with the strongest results around micro-F1 of 66.4%. The work provides a comprehensive dataset characterization, inter-annotator agreement analysis, and practical baselines, aiming to advance Portuguese claim detection in journalism and enable further NLP and IR applications for misinformation mitigation.

Abstract

Fact-checking remains a demanding and time-consuming task, still largely dependent on manual verification and unable to match the rapid spread of misinformation online. This is particularly important because debunking false information typically takes longer to reach consumers than the misinformation itself; accelerating corrections through automation can therefore help counter it more effectively. Although many organizations perform manual fact-checking, this approach is difficult to scale given the growing volume of digital content. These limitations have motivated interest in automating fact-checking, where identifying claims is a crucial first step. However, progress has been uneven across languages, with English dominating due to abundant annotated data. Portuguese, like other languages, still lacks accessible, licensed datasets, limiting research, NLP developments and applications. In this paper, we introduce ClaimPT, a dataset of European Portuguese news articles annotated for factual claims, comprising 1,308 articles and 6,875 individual annotations. Unlike most existing resources based on social media or parliamentary transcripts, ClaimPT focuses on journalistic content, collected through a partnership with LUSA, the Portuguese News Agency. To ensure annotation quality, two trained annotators labeled each article, with a curator validating all annotations according to a newly proposed scheme. We also provide baseline models for claim detection, establishing initial benchmarks and enabling future NLP and IR applications. By releasing ClaimPT, we aim to advance research on low-resource fact-checking and enhance understanding of misinformation in news media.

Figures (2)

• Figure 1: Overview of the annotation scheme.
• Figure 2: Annotation Scheme example.