Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating Actionability in Explainable AI

Gennie Mansi, Julia Kim, Mark Riedl

TL;DR

This work addresses the gap in XAI where explanations fail to clearly map to user actions in real-world settings. It employs scenario-based qualitative methods with 14 practitioners in medicine and education to build a catalog linking 12 information categories to 60 user actions, grounded in user-centered terminology. The catalog enables AI Creators to articulate expected action pathways and to test them through measurable evaluations in an exemplar system, highlighting the prominence of Mental State Actions and the role of cross-references and data provenance in actionability. The contributions offer a practical framework that expands the XAI design space by clarifying how explanations support concrete actions across domains, facilitating more usable and impactful explanations.

Abstract

A core assumption of Explainable AI (XAI) is that explanations are useful to users -- that is, users will do something with the explanations. Prior work, however, does not clearly connect the information provided in explanations to user actions to evaluate effectiveness. In this paper, we articulate this connection. We conducted a formative study through 14 interviews with end users in education and medicine. We contribute a catalog of information and associated actions. Our catalog maps 12 categories of information that participants described relying on to take 60 different actions. We show how AI Creators can use the catalog's specificity and breadth to articulate how they expect information in their explanations to lead to user actions and test their assumptions. We use an exemplar XAI system to illustrate this approach. We conclude by discussing how our catalog expands the design space for XAI systems to support actionability.

Evaluating Actionability in Explainable AI

TL;DR

This work addresses the gap in XAI where explanations fail to clearly map to user actions in real-world settings. It employs scenario-based qualitative methods with 14 practitioners in medicine and education to build a catalog linking 12 information categories to 60 user actions, grounded in user-centered terminology. The catalog enables AI Creators to articulate expected action pathways and to test them through measurable evaluations in an exemplar system, highlighting the prominence of Mental State Actions and the role of cross-references and data provenance in actionability. The contributions offer a practical framework that expands the XAI design space by clarifying how explanations support concrete actions across domains, facilitating more usable and impactful explanations.

Abstract

A core assumption of Explainable AI (XAI) is that explanations are useful to users -- that is, users will do something with the explanations. Prior work, however, does not clearly connect the information provided in explanations to user actions to evaluate effectiveness. In this paper, we articulate this connection. We conducted a formative study through 14 interviews with end users in education and medicine. We contribute a catalog of information and associated actions. Our catalog maps 12 categories of information that participants described relying on to take 60 different actions. We show how AI Creators can use the catalog's specificity and breadth to articulate how they expect information in their explanations to lead to user actions and test their assumptions. We use an exemplar XAI system to illustrate this approach. We conclude by discussing how our catalog expands the design space for XAI systems to support actionability.
Paper Structure (43 sections, 5 figures, 4 tables)

This paper contains 43 sections, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Study Procedure
  5. Recruitment
  6. Analysis
  7. Findings & Results
  8. User-Centered Action Types
  9. AI Interactions
  10. External Actions
  11. Mental State Actions
  12. User-Centered Information Categories and Associated Actions
  13. Information Categories related to Model Exposure
  14. User Variables
  15. Alternative Outcomes with the Model
  16. ...and 28 more sections

Figures (5)

  • Figure 1: Information Categories and their definitions categorized by theme.
  • Figure 2: Information Categories under Model Exposure and all actions associated with them.
  • Figure 3: Information Categories under Model Creation and all actions associated with them.
  • Figure 4: Information Categories under User Environment and all actions associated with them.
  • Figure 5: Information Categories under User Accountability and all actions associated with them.