Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Human-centered Design of Explainable Artificial Intelligence (XAI): A Survey of Empirical Studies

Shuai Ma

TL;DR

This survey briefs the technical landscape of commonly used XAI algorithms in existing empirical studies and provides an overview of the design space explored in the current human-centered XAI design, and concludes with a framework for human-centered XAI design with empirical studies.

Abstract

With the advances of AI research, AI has been increasingly adopted in numerous domains, ranging from low-stakes daily tasks such as movie recommendations to high-stakes tasks such as medicine, and criminal justice decision-making. Explainability is becoming an essential requirement for people to understand, trust and adopt AI applications. Despite a vast collection of explainable AI (XAI) algorithms produced by the AI research community, successful examples of XAI are still relatively scarce in real-world AI applications. This can be due to the gap between what the XAI is designed for and how the XAI is actually perceived by end-users. As explainability is an inherently human-centered property, in recent years, the XAI field is starting to embrace human-centered approaches and increasingly realizing the importance of empirical studies of XAI design by involving human subjects. To move a step towards a systematic review of empirical study for human-centered XAI design, in this survey, we first brief the technical landscape of commonly used XAI algorithms in existing empirical studies. Then we analyze the diverse stakeholders and needs-finding approaches. Next, we provide an overview of the design space explored in the current human-centered XAI design. Further, we summarize the evaluation metrics based on evaluation goals. Afterward, we analyze the common findings and pitfalls derived from existing studies. For each chapter, we provide a summary of current challenges and research opportunities. Finally, we conclude the survey with a framework for human-centered XAI design with empirical studies.

Towards Human-centered Design of Explainable Artificial Intelligence (XAI): A Survey of Empirical Studies

TL;DR

This survey briefs the technical landscape of commonly used XAI algorithms in existing empirical studies and provides an overview of the design space explored in the current human-centered XAI design, and concludes with a framework for human-centered XAI design with empirical studies.

Abstract

With the advances of AI research, AI has been increasingly adopted in numerous domains, ranging from low-stakes daily tasks such as movie recommendations to high-stakes tasks such as medicine, and criminal justice decision-making. Explainability is becoming an essential requirement for people to understand, trust and adopt AI applications. Despite a vast collection of explainable AI (XAI) algorithms produced by the AI research community, successful examples of XAI are still relatively scarce in real-world AI applications. This can be due to the gap between what the XAI is designed for and how the XAI is actually perceived by end-users. As explainability is an inherently human-centered property, in recent years, the XAI field is starting to embrace human-centered approaches and increasingly realizing the importance of empirical studies of XAI design by involving human subjects. To move a step towards a systematic review of empirical study for human-centered XAI design, in this survey, we first brief the technical landscape of commonly used XAI algorithms in existing empirical studies. Then we analyze the diverse stakeholders and needs-finding approaches. Next, we provide an overview of the design space explored in the current human-centered XAI design. Further, we summarize the evaluation metrics based on evaluation goals. Afterward, we analyze the common findings and pitfalls derived from existing studies. For each chapter, we provide a summary of current challenges and research opportunities. Finally, we conclude the survey with a framework for human-centered XAI design with empirical studies.

Paper Structure

This paper contains 85 sections, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Commonly Adopted XAI Techniques in Empirical Studies
  3. Global Explanation.
  4. Global feature importance
  5. Global example-based explanations
  6. Presentation of simple models
  7. Model performance
  8. Model documentation.
  9. Information about training data.
  10. Local Explanation
  11. Model uncertainty.
  12. Example-based methods.
  13. Local feature importance.
  14. Rule-based explanations.
  15. Counterfactual/Contrastive Explanations.
  16. ...and 70 more sections

Figures (4)

  • Figure 1: Examples of explanation-related design. (A) w/ and w/o explanation poursabzi2021manipulating. (B) Explanation types wang2021explanations. (C) Explanation interactivity cheng2019explaining. (D) Granularity/complexity of explanation abdul2020cogam. (E) Modality of explanation szymanski2021visual. (F) Explanation adaptability bansal2021does.
  • Figure 2: Model and prediction-related design. (A) W/ and W/o Tutorial/Training lai2020chicago. (B) Explanation of Training Data anik2021data. (C) W/ and W/o Confidence/Uncertainty bansal2021does. (D) Model performance yin2019understanding.
  • Figure 3: Human-AI collaboration mode-related design. (A) AI Agency (Degree of Explanation) lai2019human. (B) Expertise of Explanation Users szymanski2021visual. (C) Cognitive Bias Mitigating buccinca2021trust.
  • Figure 4: A framework of human-centered XAI design.