TSVer: A Benchmark for Fact Verification Against Time-Series Evidence
Marek Strong, Andreas Vlachos
TL;DR
TSVer introduces the first benchmark for fact verification grounded in real-world time-series evidence, addressing the need for robust numerical and temporal reasoning in claim verification. It builds a dataset of 287 claims from 38 fact-checking organizations and 400 time series from OWID, with time-frame annotations, verdicts, and structured justifications produced via an LLM-assisted two-round annotation process achieving substantial inter-annotator agreement. The authors propose a baseline retrieval-and-verification pipeline and evaluate state-of-the-art models (Gemini-2.5-Pro, GPT-4) against TSVer, highlighting substantial challenges in time-series reasoning and retrieval efficiency; Ev2R and TSCS metrics reveal gaps in justification quality and evidence coverage. The work demonstrates the dataset's potential to drive explainable fact verification research and outlines limitations like multilingual coverage, source bias, and the scope of evidence. TSVer provides a benchmark, baseline tools, and evaluation metrics to spur progress in reliable, explainable verification using structured time-series data.
Abstract
Reasoning over temporal and numerical data, such as time series, is a crucial aspect of fact-checking. While many systems have recently been developed to handle this form of evidence, their evaluation remains limited by existing datasets, which often lack structured evidence, provide insufficient justifications for verdicts, or rely on synthetic claims. In this paper, we introduce TSVer, a new benchmark dataset for fact verification focusing on temporal and numerical reasoning with time-series evidence. TSVer contains 287 real-world claims sourced from 38 fact-checking organizations and a curated database of 400 time series covering diverse domains. Each claim is annotated with time frames across all pertinent time series, along with a verdict and justifications reflecting how the evidence is used to reach the verdict. Using an LLM-assisted multi-step annotation process, we improve the quality of our annotations and achieve an inter-annotator agreement of kappa=0.745 on verdicts. We also develop a baseline for verifying claims against time-series evidence and show that even the state-of-the-art reasoning models like Gemini-2.5-Pro are challenged by time series, achieving a 63.37 accuracy score on verdicts and an Ev2R score of 48.63 on verdict justifications.