The CLEF-2025 CheckThat! Lab: Subjectivity, Fact-Checking, Claim Normalization, and Retrieval
Firoj Alam, Julia Maria Struß, Tanmoy Chakraborty, Stefan Dietze, Salim Hafid, Katerina Korre, Arianna Muti, Preslav Nakov, Federico Ruggeri, Sebastian Schellhammer, Vinay Setty, Megha Sundriyal, Konstantin Todorov, Venktesh V
TL;DR
The paper outlines the CLEF-2025 CheckThat! Lab, detailing four tasks that advance the information verification pipeline across multiple languages: Task 1 subjectivity detection to refine check-worthiness; Task 2 claim normalization to generate concise, verifiable assertions from noisy posts; Task 3 numerical-claim verification to classify claims as True, False, or Conflicting using BM25-derived evidence; and Task 4 SciWeb processing for detecting scientific discourse and retrieving the corresponding sources. It emphasizes a multilingual, data-rich setup leveraging sources like the Google Fact-check Explorer API, Claim Review Schema, SciTweets, Altmetric, and the CORD-19 corpus, with evaluation metrics including macro-$F_1$, METEOR, and $MRR@5$. The contribution is a coherent, openly available framework of datasets, baselines, and evaluation protocols across 20 languages, aiming to streamline verification workflows and support research decision-making. The work highlights the practical impact of scalable, language-diverse tools for countering online misinformation and supporting journalists and researchers in evidence-based reporting.
Abstract
The CheckThat! lab aims to advance the development of innovative technologies designed to identify and counteract online disinformation and manipulation efforts across various languages and platforms. The first five editions focused on key tasks in the information verification pipeline, including check-worthiness, evidence retrieval and pairing, and verification. Since the 2023 edition, the lab has expanded its scope to address auxiliary tasks that support research and decision-making in verification. In the 2025 edition, the lab revisits core verification tasks while also considering auxiliary challenges. Task 1 focuses on the identification of subjectivity (a follow-up from CheckThat! 2024), Task 2 addresses claim normalization, Task 3 targets fact-checking numerical claims, and Task 4 explores scientific web discourse processing. These tasks present challenging classification and retrieval problems at both the document and span levels, including multilingual settings.