Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TELLER: A Trustworthy Framework for Explainable, Generalizable and Controllable Fake News Detection

Hui Liu, Wenya Wang, Haoru Li, Haoliang Li

TL;DR

Teller addresses the trustworthiness gap in fake news detection by decoupling the problem into a cognition system that encodes human expertise into interpretable predicates and a decision system that learns generalizable rules through a neural-symbolic DNF Layer. The cognition system uses question templates and logic atoms answered by LLMs and tools, while the decision system aggregates these atoms into domain-invariant rules, enabling robust cross-domain performance and human controllability. Across four datasets and multiple LLMs, Teller demonstrates strong accuracy and macro-F1, with explicit explainability via extractable rules and pruning, and controllability through rule adjustments and cognition-system interventions. This framework has practical impact for trustworthy deployment of fake news detectors by combining scalable AI with transparent reasoning and human oversight, reducing reliance on opaque models. Future work may enhance the semantic grounding of predicates and broaden the decision model to further improve interpretability, controllability, and lifecyle trustworthiness.

Abstract

The proliferation of fake news has emerged as a severe societal problem, raising significant interest from industry and academia. While existing deep-learning based methods have made progress in detecting fake news accurately, their reliability may be compromised caused by the non-transparent reasoning processes, poor generalization abilities and inherent risks of integration with large language models (LLMs). To address this challenge, we propose {\methodname}, a novel framework for trustworthy fake news detection that prioritizes explainability, generalizability and controllability of models. This is achieved via a dual-system framework that integrates cognition and decision systems, adhering to the principles above. The cognition system harnesses human expertise to generate logical predicates, which guide LLMs in generating human-readable logic atoms. Meanwhile, the decision system deduces generalizable logic rules to aggregate these atoms, enabling the identification of the truthfulness of the input news across diverse domains and enhancing transparency in the decision-making process. Finally, we present comprehensive evaluation results on four datasets, demonstrating the feasibility and trustworthiness of our proposed framework. Our implementation is available at \url{https://github.com/less-and-less-bugs/Trust_TELLER}.

TELLER: A Trustworthy Framework for Explainable, Generalizable and Controllable Fake News Detection

TL;DR

Teller addresses the trustworthiness gap in fake news detection by decoupling the problem into a cognition system that encodes human expertise into interpretable predicates and a decision system that learns generalizable rules through a neural-symbolic DNF Layer. The cognition system uses question templates and logic atoms answered by LLMs and tools, while the decision system aggregates these atoms into domain-invariant rules, enabling robust cross-domain performance and human controllability. Across four datasets and multiple LLMs, Teller demonstrates strong accuracy and macro-F1, with explicit explainability via extractable rules and pruning, and controllability through rule adjustments and cognition-system interventions. This framework has practical impact for trustworthy deployment of fake news detectors by combining scalable AI with transparent reasoning and human oversight, reducing reliance on opaque models. Future work may enhance the semantic grounding of predicates and broaden the decision model to further improve interpretability, controllability, and lifecyle trustworthiness.

Abstract

The proliferation of fake news has emerged as a severe societal problem, raising significant interest from industry and academia. While existing deep-learning based methods have made progress in detecting fake news accurately, their reliability may be compromised caused by the non-transparent reasoning processes, poor generalization abilities and inherent risks of integration with large language models (LLMs). To address this challenge, we propose {\methodname}, a novel framework for trustworthy fake news detection that prioritizes explainability, generalizability and controllability of models. This is achieved via a dual-system framework that integrates cognition and decision systems, adhering to the principles above. The cognition system harnesses human expertise to generate logical predicates, which guide LLMs in generating human-readable logic atoms. Meanwhile, the decision system deduces generalizable logic rules to aggregate these atoms, enabling the identification of the truthfulness of the input news across diverse domains and enhancing transparency in the decision-making process. Finally, we present comprehensive evaluation results on four datasets, demonstrating the feasibility and trustworthiness of our proposed framework. Our implementation is available at \url{https://github.com/less-and-less-bugs/Trust_TELLER}.
Paper Structure (30 sections, 5 equations, 2 figures, 16 tables, 2 algorithms)

This paper contains 30 sections, 5 equations, 2 figures, 16 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Trustworthy AI
  4. Trustworthy Fake News Detection
  5. Methodology
  6. Cognition System
  7. Predicate Construction
  8. Logic evaluation with LLMs
  9. Decision System
  10. Experiments
  11. Experimental Setting
  12. Feasibility Study
  13. Explainability Verification
  14. Generalizability Verification
  15. Controllability Verification
  16. ...and 15 more sections

Figures (2)

  • Figure 1: Three crucial aspects of trustworthy fake news detection algorithms and the correlation between these principles and our dual-sytem framework Teller.
  • Figure 2: The architecture of the proposed framework Teller. $N$ represents the number of question templates (logic predicates), $M_i$ denotes the number of logic atoms corresponding to the $i$th predicate, $\mathcal{Y}$ denotes the truthfulness label set. The semantics of question templates and logic predicates are described in Table \ref{['tab:logic-table']}.