POLygraph: Polish Fake News Dataset
Daniel Dzienisiewicz, Filip Graliński, Piotr Jabłoński, Marek Kubis, Paweł Skórzewski, Piotr Wierzchoń
TL;DR
POLygraph delivers a dual-part Polish fake-news dataset, combining the fake-or-not (11,360 URL-labeled article pairs) and fake-they-say (5,082 articles with tweet-level opinions) datasets to support multi-signal fake-news detection. It details a comprehensive data-collection pipeline (web scraping and API access), a two-pronged annotation framework (fake-or-not and fake-they-say) with extensive human-in-the-loop labeling, and an accompanying privatization tool to protect privacy. While the paper centers on dataset construction rather than tool evaluation, it provides an end-to-end resource architecture (Doccano/Gonito.net/GEval) and a scalable benchmarking setup for cross-task fake-news research in Polish and potentially other languages. The POLygraph suite has practical implications for public sector monitoring, media analysis, and fact-checking workflows, and lays groundwork for extending analogous datasets to additional languages.
Abstract
This paper presents the POLygraph dataset, a unique resource for fake news detection in Polish. The dataset, created by an interdisciplinary team, is composed of two parts: the "fake-or-not" dataset with 11,360 pairs of news articles (identified by their URLs) and corresponding labels, and the "fake-they-say" dataset with 5,082 news articles (identified by their URLs) and tweets commenting on them. Unlike existing datasets, POLygraph encompasses a variety of approaches from source literature, providing a comprehensive resource for fake news detection. The data was collected through manual annotation by expert and non-expert annotators. The project also developed a software tool that uses advanced machine learning techniques to analyze the data and determine content authenticity. The tool and dataset are expected to benefit various entities, from public sector institutions to publishers and fact-checking organizations. Further dataset exploration will foster fake news detection and potentially stimulate the implementation of similar models in other languages. The paper focuses on the creation and composition of the dataset, so it does not include a detailed evaluation of the software tool for content authenticity analysis, which is planned at a later stage of the project.