Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Why do explanations fail? A typology and discussion on failures in XAI

Clara Bove, Thibault Laugel, Marie-Jeanne Lesot, Charles Tijus, Marcin Detyniecki

TL;DR

This paper introduces a holistic typology of XAI failures by separating system-specific (technical) failures from user-specific (interpretation) failures, addressing observed fragmentation in prior work. It systematically builds a taxonomy from literature (n=108 papers) using a meta-characteristic framework and ends conditions, capturing how failures arise and propagate through model, explainer, and user interactions. The typology distinguishes misleading, competing, unstable, and incompatible system-level failures, as well as mismatches, counterintuitive explanations, and biased inferences at the user level, and discusses mitigation strategies across design, evaluation, and interfaces. The authors advocate for holistic, user-centered, and transparent XAI design, stressing the need for integrative approaches that consider interactions among components and richer XUI designs to improve explainability and trust in AI systems. Overall, the work provides a structured lens to diagnose XAI limitations and map research directions toward more robust, personalized, and comprehensible explanations with practical implications for practitioners and researchers.

Abstract

As Machine Learning models achieve unprecedented levels of performance, the XAI domain aims at making these models understandable by presenting end-users with intelligible explanations. Yet, some existing XAI approaches fail to meet expectations: several issues have been reported in the literature, generally pointing out either technical limitations or misinterpretations by users. In this paper, we argue that the resulting harms arise from a complex overlap of multiple failures in XAI, which existing ad-hoc studies fail to capture. This work therefore advocates for a holistic perspective, presenting a systematic investigation of limitations of current XAI methods and their impact on the interpretation of explanations. % By distinguishing between system-specific and user-specific failures, we propose a typological framework that helps revealing the nuanced complexities of explanation failures. Leveraging this typology, we discuss some research directions to help practitioners better understand the limitations of XAI systems and enhance the quality of ML explanations.

Why do explanations fail? A typology and discussion on failures in XAI

TL;DR

This paper introduces a holistic typology of XAI failures by separating system-specific (technical) failures from user-specific (interpretation) failures, addressing observed fragmentation in prior work. It systematically builds a taxonomy from literature (n=108 papers) using a meta-characteristic framework and ends conditions, capturing how failures arise and propagate through model, explainer, and user interactions. The typology distinguishes misleading, competing, unstable, and incompatible system-level failures, as well as mismatches, counterintuitive explanations, and biased inferences at the user level, and discusses mitigation strategies across design, evaluation, and interfaces. The authors advocate for holistic, user-centered, and transparent XAI design, stressing the need for integrative approaches that consider interactions among components and richer XUI designs to improve explainability and trust in AI systems. Overall, the work provides a structured lens to diagnose XAI limitations and map research directions toward more robust, personalized, and comprehensible explanations with practical implications for practitioners and researchers.

Abstract

As Machine Learning models achieve unprecedented levels of performance, the XAI domain aims at making these models understandable by presenting end-users with intelligible explanations. Yet, some existing XAI approaches fail to meet expectations: several issues have been reported in the literature, generally pointing out either technical limitations or misinterpretations by users. In this paper, we argue that the resulting harms arise from a complex overlap of multiple failures in XAI, which existing ad-hoc studies fail to capture. This work therefore advocates for a holistic perspective, presenting a systematic investigation of limitations of current XAI methods and their impact on the interpretation of explanations. % By distinguishing between system-specific and user-specific failures, we propose a typological framework that helps revealing the nuanced complexities of explanation failures. Leveraging this typology, we discuss some research directions to help practitioners better understand the limitations of XAI systems and enhance the quality of ML explanations.
Paper Structure (59 sections, 2 figures, 1 table)

This paper contains 59 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Context and Related Works
  3. Explanation Process Components
  4. Notion of Explanation Failure
  5. Related Works
  6. Research Questions and Methodology
  7. Research Questions
  8. RQ1: What are the different failures that may arise during the explanation process?
  9. RQ2: How can these failures be avoided?
  10. Methodology
  11. Meta-characteristic
  12. Ending conditions
  13. Data collection
  14. Overview of the Result
  15. Proposed Typology: System-specific XAI Failures
  16. ...and 44 more sections

Figures (2)

  • Figure 1: Types of system-specific explanation failures: misleading explanations ascribed to the ML system and contradicting ones ascribed to the explainer. The cross and the $\neq$ symbol indicate the step at which the failure takes place.
  • Figure 2: Types of user-specific explanation failures in ML explanations. The ML model is supposed to give an accurate prediction and the explainer to generate faithful explanations. The cross indicates the step at which the failure takes place.