Fact-Checking Complex Claims with Program-Guided Reasoning
Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, Preslav Nakov
TL;DR
ProgramFC introduces a few-shot neuro-symbolic framework for fact-checking that decomposes complex claims into executable reasoning programs. A Codex-based program generator creates a sequence of sub-task calls, which are then executed by specialized functions (Question, Verify, Predict) to produce a veracity label and explanations. The approach delivers explainability and data efficiency, achieving state-of-the-art results on multi-hop datasets (HOVER, FEVEROUS) with strong performance gains for deeper reasoning and improved evidence retrieval through iterative retrieval. The work also analyzes interpretability, error modes, and limitations, such as computational cost, while outlining future directions for broader applicability and multi-modal reasoning.
Abstract
Fact-checking real-world claims often requires collecting multiple pieces of evidence and applying complex multi-step reasoning. In this paper, we present Program-Guided Fact-Checking (ProgramFC), a novel fact-checking model that decomposes complex claims into simpler sub-tasks that can be solved using a shared library of specialized functions. We first leverage the in-context learning ability of large language models to generate reasoning programs to guide the verification process. Afterward, we execute the program by delegating each sub-task to the corresponding sub-task handler. This process makes our model both explanatory and data-efficient, providing clear explanations of its reasoning process and requiring minimal training data. We evaluate ProgramFC on two challenging fact-checking datasets and show that it outperforms seven fact-checking baselines across different settings of evidence availability, with explicit output programs that benefit human debugging. Our codes and data are publicly available at https://github.com/mbzuai-nlp/ProgramFC.