One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques

TL;DR

The paper tackles the mismatch between diverse societal demands for AI explanations and the variety of explainability methods available. It presents a taxonomy and an open-source toolkit, AIX360, comprising eight explainability algorithms and two metrics, organized to support data-, model-, and prediction-level explanations across local/global scopes. Enhanced by algorithmic simplifications, data-synthesis methods, visualization capabilities, and educational resources (web demo and tutorials), the work aims to bridge research and real-world deployment. By mapping methods to concrete consumer personas and providing a unified API, it enables practitioners to select appropriate explanations, identifies gaps for further development, and promotes broader adoption of explainable AI.

Abstract

As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360 (http://aix360.mybluemix.net/), an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. We also discuss enhancements to bring research innovations closer to consumers of explanations, ranging from simplified, more accessible versions of algorithms, to tutorials and an interactive web demo to introduce AI explainability to different audiences and application domains. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.

Figures (7)

• Figure 1: The proposed taxonomy based on questions about what is explained (e.g., data or model), how it is explained (e.g., direct/post-hoc, static/interactive) and at what level (i.e. local/global). The decision tree leaves indicate the methods currently available through the toolkit along with the modalities they can work with. '?' indicates that the toolkit currently does not have a method available in the particular category.
• Figure 2: Organization of AIX360 explainer classes according to their use in various steps of the AI modeling pipeline.
• Figure 3: Rule complexity-test accuracy trade-offs. Pareto efficient points are connected by line segments. Horizontal and vertical bars represent standard errors in the means.
• Figure 4: Rule complexity-test accuracy trade-offs. Pareto efficient points are connected by line segments. Horizontal and vertical bars represent standard errors in the means.
• Figure 5: Rule complexity-test accuracy trade-offs. Pareto efficient points are connected by line segments. Horizontal and vertical bars represent standard errors in the means.