Defining Explainable AI for Requirements Analysis
Raymond Sheh, Isaac Monteath
TL;DR
The paper addresses the challenge of defining concrete explanatory requirements for AI explanations across diverse applications. It introduces a three-dimensional taxonomy—Source, Depth, and Scope—to categorize explanations and maps these categories to ML techniques. It demonstrates through UX, forensic/compliance, neural networks, and decision-tree examples how different explanations serve different needs, including post-hoc rationalisation, introspective, attribute-based, and model-based explanations. The proposed framework enables explicit trade-offs between explainability, accuracy, and adaptability and points to future work on measurement, verifiability, and repairability.
Abstract
Explainable Artificial Intelligence (XAI) has become popular in the last few years. The Artificial Intelligence (AI) community in general, and the Machine Learning (ML) community in particular, is coming to the realisation that in many applications, for AI to be trusted, it must not only demonstrate good performance in its decisionmaking, but it also must explain these decisions and convince us that it is making the decisions for the right reasons. However, different applications have different requirements on the information required of the underlying AI system in order to convince us that it is worthy of our trust. How do we define these requirements? In this paper, we present three dimensions for categorising the explanatory requirements of different applications. These are Source, Depth and Scope. We focus on the problem of matching up the explanatory requirements of different applications with the capabilities of underlying ML techniques to provide them. We deliberately avoid including aspects of explanation that are already well-covered by the existing literature and we focus our discussion on ML although the principles apply to AI more broadly.