Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking
Greta Warren, Irina Shklovski, Isabelle Augenstein
TL;DR
This paper investigates how fact-checkers define and demand explanations from automated fact-checking systems. Using semi-structured interviews with 10 professionals across five continents, it maps decision-making across claim detection, evidence retrieval, verdict, and communication, revealing substantial gaps between current AI explainability and practitioner needs. The authors identify core requirements such as replicability, local, evidence-linked explanations, and primary-source emphasis, and offer design implications to align tools with human workflows. They argue for human-centered, transparent, and ethically aware development to enhance the utility and trustworthiness of automated fact-checking in real-world journalism.
Abstract
The pervasiveness of large language models and generative AI in online media has amplified the need for effective automated fact-checking to assist fact-checkers in tackling the increasing volume and sophistication of misinformation. The complex nature of fact-checking demands that automated fact-checking systems provide explanations that enable fact-checkers to scrutinise their outputs. However, it is unclear how these explanations should align with the decision-making and reasoning processes of fact-checkers to be effectively integrated into their workflows. Through semi-structured interviews with fact-checking professionals, we bridge this gap by: (i) providing an account of how fact-checkers assess evidence, make decisions, and explain their processes; (ii) examining how fact-checkers use automated tools in practice; and (iii) identifying fact-checker explanation requirements for automated fact-checking tools. The findings show unmet explanation needs and identify important criteria for replicable fact-checking explanations that trace the model's reasoning path, reference specific evidence, and highlight uncertainty and information gaps.