Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

TL;DR

Addressing explainability in deep learning, the paper surveys XAI methods through a structured taxonomy along scope, methodology, and usage. It provides mathematical summaries of seminal works, a historical timeline from 2007 to 2020, and an evaluation of explanation maps produced by eight XAI algorithms on image data. The authors discuss evaluation methods, limitations of current approaches, and future directions to enhance trust, transparency, and fairness in AI. They also offer practical guidance, including software resources, to support researchers and practitioners in applying XAI techniques.

Abstract

Nowadays, deep neural networks are widely used in mission critical systems such as healthcare, self-driving vehicles, and military which have direct impact on human lives. However, the black-box nature of deep neural networks challenges its use in mission critical applications, raising ethical and judicial concerns inducing lack of trust. Explainable Artificial Intelligence (XAI) is a field of Artificial Intelligence (AI) that promotes a set of tools, techniques, and algorithms that can generate high-quality interpretable, intuitive, human-understandable explanations of AI decisions. In addition to providing a holistic view of the current XAI landscape in deep learning, this paper provides mathematical summaries of seminal work. We start by proposing a taxonomy and categorizing the XAI techniques based on their scope of explanations, methodology behind the algorithms, and explanation level or usage which helps build trustworthy, interpretable, and self-explanatory deep learning models. We then describe the main principles used in XAI research and present the historical timeline for landmark studies in XAI from 2007 to 2020. After explaining each category of algorithms and approaches in detail, we then evaluate the explanation maps generated by eight XAI algorithms on image data, discuss the limitations of this approach, and provide potential future directions to improve XAI evaluation.

Figures (4)

• Figure 1: General categorization of the survey in terms of scope, methodology, and usage.
• Figure 2: High-level illustration of deep learning model $f$. Generally, a single input instance $x$ generates outputs $\Bar{y}$. No other metadata or explanations are generated other than the output classification. Most model inference scenarios involve this method where model $f$ is considered as a blob of information which takes an input $x$ and generates an output $\Bar{y}$
• Figure 3: Illustration from Goodfellow2015 showing an adversarial attack where an image class Panda is deliberately attacked to predict as a Gibbon with high confidence. Note that the attacked image is visually similar to the original image and humans are unable to understand any changes.
• Figure 4: Illustration from Lapuschkin2019 showing how text in images can fool classifiers into believing that the text is a feature for a particular task.