Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Explainable Framework for Misinformation Identification via Critical Question Answering

Ramon Ruiz-Dolz, John Lawrence

TL;DR

This work tackles the lack of explainability in misinformation detection by integrating Walton's Argumentation Schemes and Critical Questions into an NLP framework. It introduces a two-stage pipeline (Argumentation Scheme Classification and Critical Question Answering) and the NLAS-CQ corpus to support explainable reasoning about misinformation. The framework not only detects potentially misleading arguments but also provides structured, human-interpretable explanations via critical questions, validated with strong performance on textbook-like arguments and informative baselines for CQA. The release of NLAS-CQ and the demonstrated approach offer a principled path toward automated, explainable reason-checking in NLP applications.

Abstract

Natural language misinformation detection approaches have been, to date, largely dependent on sequence classification methods, producing opaque systems in which the reasons behind classification as misinformation are unclear. While an effort has been made in the area of automated fact-checking to propose explainable approaches to the problem, this is not the case for automated reason-checking systems. In this paper, we propose a new explainable framework for both factual and rational misinformation detection based on the theory of Argumentation Schemes and Critical Questions. For that purpose, we create and release NLAS-CQ, the first corpus combining 3,566 textbook-like natural language argumentation scheme instances and 4,687 corresponding answers to critical questions related to these arguments. On the basis of this corpus, we implement and validate our new framework which combines classification with question answering to analyse arguments in search of misinformation, and provides the explanations in form of critical questions to the human user.

An Explainable Framework for Misinformation Identification via Critical Question Answering

TL;DR

This work tackles the lack of explainability in misinformation detection by integrating Walton's Argumentation Schemes and Critical Questions into an NLP framework. It introduces a two-stage pipeline (Argumentation Scheme Classification and Critical Question Answering) and the NLAS-CQ corpus to support explainable reasoning about misinformation. The framework not only detects potentially misleading arguments but also provides structured, human-interpretable explanations via critical questions, validated with strong performance on textbook-like arguments and informative baselines for CQA. The release of NLAS-CQ and the demonstrated approach offer a principled path toward automated, explainable reason-checking in NLP applications.

Abstract

Natural language misinformation detection approaches have been, to date, largely dependent on sequence classification methods, producing opaque systems in which the reasons behind classification as misinformation are unclear. While an effort has been made in the area of automated fact-checking to propose explainable approaches to the problem, this is not the case for automated reason-checking systems. In this paper, we propose a new explainable framework for both factual and rational misinformation detection based on the theory of Argumentation Schemes and Critical Questions. For that purpose, we create and release NLAS-CQ, the first corpus combining 3,566 textbook-like natural language argumentation scheme instances and 4,687 corresponding answers to critical questions related to these arguments. On the basis of this corpus, we implement and validate our new framework which combines classification with question answering to analyse arguments in search of misinformation, and provides the explanations in form of critical questions to the human user.

Paper Structure

This paper contains 20 sections, 2 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Framework
  4. The NLAS-CQ Corpus
  5. Argumentation Scheme Generation
  6. Critical Question Annotation
  7. Experiments
  8. Experimental Setup
  9. Results
  10. Argumentation Scheme Classification (ASC)
  11. Critical Question Answering (CQA)
  12. Error Analysis
  13. Conclusion
  14. Details of the NLAS-CQ Corpus
  15. Argument Distribution per Argumentation Scheme
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Critical Question-based Explainable Framework for Misinformation Detection.
  • Figure 2: Aggregated confusion matrices of the CQA models (3 runs). Rows represent ground-truth values and columns the predicted answer to the CQs.
  • Figure 3: Distribution of the RoBERTa-cqa [ci] model outputs in the CQA test data. Green represents correct answers while red represents incorrect answers.
  • Figure 4: Distribution of the CQ answers in the complete NLAS-CQ corpus. Blue represents affirmative answers while pink presents negative answers.