Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explainable Fake News Detection With Large Language Model via Defense Among Competing Wisdom

Bo Wang, Jing Ma, Hongzhan Lin, Zhiwei Yang, Ruichao Yang, Yuan Tian, Yi Chang

TL;DR

This work tackles the explainability challenge in fake news detection by introducing L-Defense, a defense-based framework that leverages competing wisdom within raw reports. It decomposes the problem into competing evidence extraction, prompt-based LLM reasoning to generate veracity-oriented explanations, and a defense-based inference step that selects the verdict by assessing the relative strength of two competing justifications. The approach mitigates majority-bias and reduces reliance on debunked sources, achieving state-of-the-art or competitive results on RAWFC and LIAR-RAW while delivering interpretable rationales. Human and automated evaluation of explanations further demonstrates the quality and coherence of the generated justifications, underscoring the practical impact for timely, transparent fake news detection.

Abstract

Most fake news detection methods learn latent feature representations based on neural networks, which makes them black boxes to classify a piece of news without giving any justification. Existing explainable systems generate veracity justifications from investigative journalism, which suffer from debunking delayed and low efficiency. Recent studies simply assume that the justification is equivalent to the majority opinions expressed in the wisdom of crowds. However, the opinions typically contain some inaccurate or biased information since the wisdom of crowds is uncensored. To detect fake news from a sea of diverse, crowded and even competing narratives, in this paper, we propose a novel defense-based explainable fake news detection framework. Specifically, we first propose an evidence extraction module to split the wisdom of crowds into two competing parties and respectively detect salient evidences. To gain concise insights from evidences, we then design a prompt-based module that utilizes a large language model to generate justifications by inferring reasons towards two possible veracities. Finally, we propose a defense-based inference module to determine veracity via modeling the defense among these justifications. Extensive experiments conducted on two real-world benchmarks demonstrate that our proposed method outperforms state-of-the-art baselines in terms of fake news detection and provides high-quality justifications.

Explainable Fake News Detection With Large Language Model via Defense Among Competing Wisdom

TL;DR

This work tackles the explainability challenge in fake news detection by introducing L-Defense, a defense-based framework that leverages competing wisdom within raw reports. It decomposes the problem into competing evidence extraction, prompt-based LLM reasoning to generate veracity-oriented explanations, and a defense-based inference step that selects the verdict by assessing the relative strength of two competing justifications. The approach mitigates majority-bias and reduces reliance on debunked sources, achieving state-of-the-art or competitive results on RAWFC and LIAR-RAW while delivering interpretable rationales. Human and automated evaluation of explanations further demonstrates the quality and coherence of the generated justifications, underscoring the practical impact for timely, transparent fake news detection.

Abstract

Most fake news detection methods learn latent feature representations based on neural networks, which makes them black boxes to classify a piece of news without giving any justification. Existing explainable systems generate veracity justifications from investigative journalism, which suffer from debunking delayed and low efficiency. Recent studies simply assume that the justification is equivalent to the majority opinions expressed in the wisdom of crowds. However, the opinions typically contain some inaccurate or biased information since the wisdom of crowds is uncensored. To detect fake news from a sea of diverse, crowded and even competing narratives, in this paper, we propose a novel defense-based explainable fake news detection framework. Specifically, we first propose an evidence extraction module to split the wisdom of crowds into two competing parties and respectively detect salient evidences. To gain concise insights from evidences, we then design a prompt-based module that utilizes a large language model to generate justifications by inferring reasons towards two possible veracities. Finally, we propose a defense-based inference module to determine veracity via modeling the defense among these justifications. Extensive experiments conducted on two real-world benchmarks demonstrate that our proposed method outperforms state-of-the-art baselines in terms of fake news detection and provides high-quality justifications.
Paper Structure (38 sections, 16 equations, 3 figures, 11 tables)

This paper contains 38 sections, 16 equations, 3 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Explainable Fake News Detection
  4. Large Language Model in Fake News Detection
  5. Proposed Approach
  6. Task Definition.
  7. Competing Evidence Extraction
  8. Claim and Evidence Representation
  9. Competing Veracity Scoring
  10. Training and Inference
  11. Training.
  12. Inference.
  13. Prompt-based Reasoning with LLM
  14. Defense-based Inference
  15. Discussion about Generalization
  16. ...and 23 more sections

Figures (3)

  • Figure 1: A false claim from the Sina Weibo. The comparison of informativeness and soundness between two competing parties serves as an indicator of veracity.
  • Figure 2: An overview of the proposed LLM-equipped defense-based explainable fake news detection (L-Defense) framework.
  • Figure 3: Confusion matrixes of the judgment results made by 10 annotators on 30 randomly sampled samples. The left one is derived by providing annotators with only the claim, while the right one is derived by providing annotators with both the claim and the explanations generated by our L-Defense. The results from 10 annotators were averaged and rounded off.