Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generating Zero-shot Abstractive Explanations for Rumour Verification

Iman Munire Bilal, Preslav Nakov, Rob Procter, Maria Liakata

TL;DR

This work tackles generating natural language explanations for rumour veracity rather than solely predicting labels. It introduces a zero-shot, model-agnostic pipeline that pairs a structure- and stance-aware GNN verifier with attribution-based post selection and abstractive summarisation to produce informative explanations. Abstractive, model-centric summaries derived from the most important posts outperform extractive and model-independent baselines, and IG-based attributions offer better fidelity and efficiency than Shapley Values. The authors additionally validate explanations with LLM-based evaluators, finding substantial agreement with human judgments and demonstrating scalable evaluation for complex, real-world rumour scenarios. Overall, the approach advances explainable rumour verification and highlights practical avenues for scalable evaluation using LLMs.

Abstract

The task of rumour verification in social media concerns assessing the veracity of a claim on the basis of conversation threads that result from it. While previous work has focused on predicting a veracity label, here we reformulate the task to generate model-centric free-text explanations of a rumour's veracity. The approach is model agnostic in that it generalises to any model. Here we propose a novel GNN-based rumour verification model. We follow a zero-shot approach by first applying post-hoc explainability methods to score the most important posts within a thread and then we use these posts to generate informative explanations using opinion-guided summarisation. To evaluate the informativeness of the explanatory summaries, we exploit the few-shot learning capabilities of a large language model (LLM). Our experiments show that LLMs can have similar agreement to humans in evaluating summaries. Importantly, we show explanatory abstractive summaries are more informative and better reflect the predicted rumour veracity than just using the highest ranking posts in the thread.

Generating Zero-shot Abstractive Explanations for Rumour Verification

TL;DR

This work tackles generating natural language explanations for rumour veracity rather than solely predicting labels. It introduces a zero-shot, model-agnostic pipeline that pairs a structure- and stance-aware GNN verifier with attribution-based post selection and abstractive summarisation to produce informative explanations. Abstractive, model-centric summaries derived from the most important posts outperform extractive and model-independent baselines, and IG-based attributions offer better fidelity and efficiency than Shapley Values. The authors additionally validate explanations with LLM-based evaluators, finding substantial agreement with human judgments and demonstrating scalable evaluation for complex, real-world rumour scenarios. Overall, the approach advances explainable rumour verification and highlights practical avenues for scalable evaluation using LLMs.

Abstract

The task of rumour verification in social media concerns assessing the veracity of a claim on the basis of conversation threads that result from it. While previous work has focused on predicting a veracity label, here we reformulate the task to generate model-centric free-text explanations of a rumour's veracity. The approach is model agnostic in that it generalises to any model. Here we propose a novel GNN-based rumour verification model. We follow a zero-shot approach by first applying post-hoc explainability methods to score the most important posts within a thread and then we use these posts to generate informative explanations using opinion-guided summarisation. To evaluate the informativeness of the explanatory summaries, we exploit the few-shot learning capabilities of a large language model (LLM). Our experiments show that LLMs can have similar agreement to humans in evaluating summaries. Importantly, we show explanatory abstractive summaries are more informative and better reflect the predicted rumour veracity than just using the highest ranking posts in the thread.
Paper Structure (37 sections, 1 equation, 2 figures, 9 tables)

This paper contains 37 sections, 1 equation, 2 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Explainable Fact Checking
  4. Explainability as Summarisation
  5. LLMs as evaluators
  6. Methodology
  7. Task Formulation
  8. Rumour Verification Model
  9. Structure-Aware Model
  10. Stance-Aware Component
  11. Performance of Rumour Verification Baselines
  12. Explaining the Model
  13. Attribution Method
  14. Summarisation for Explanation
  15. Extractive Explanations
  16. ...and 22 more sections

Figures (2)

  • Figure 1: Framework of our proposed approach to obtain faithful generated explanations for the rumour verification model. It explains the process of explanation generation, where the weights from a model are passed through an explainer algorithm to identify important input nodes, which are then filtered and used in abstractive summarisation.
  • Figure 2: Architecture of our rumour verification model enhanced with structure-aware and stance-aware components based on graph neural networks. In the diagram, Propagation/Dispersion/Dispersion represent the outputs of each respective component, while Propagation*/Dispersion* represent the stance-enriched outputs of these.