Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

User Decision Guidance with Selective Explanation Presentation from Explainable-AI

Yosuke Fukuchi, Seiji Yamada

TL;DR

This work tackles how to select explanations in explainable AI–driven decision support by introducing X-Selector, a method that predicts how different explanation combinations influence user decisions and selects the set that aligns with the AI's suggested decision. It leverages a UserModel to anticipate user responses and a policy π to characterize AI decisions, optimizing the match between human and AI choices via a discrepancy-minimizing objective. Through a stock-trading simulation with StockAI, the study demonstrates that selective explanations can outperform naive strategies, particularly when AI accuracy is high, while revealing limitations in low-accuracy contexts where AI unreliability reduces the benefit of guided explanations. These findings highlight the potential and limits of libertarian-paternalism-inspired explanation design in real-world IDSSs and point to future work on calibrating guidance strength and incorporating historical explanation context.

Abstract

This paper addresses the challenge of selecting explanations for XAI (Explainable AI)-based Intelligent Decision Support Systems (IDSSs). IDSSs have shown promise in improving user decisions through XAI-generated explanations along with AI predictions, and the development of XAI made it possible to generate a variety of such explanations. However, how IDSSs should select explanations to enhance user decision-making remains an open question. This paper proposes X-Selector, a method for selectively presenting XAI explanations. It enables IDSSs to strategically guide users to an AI-suggested decision by predicting the impact of different combinations of explanations on a user's decision and selecting the combination that is expected to minimize the discrepancy between an AI suggestion and a user decision. We compared the efficacy of X-Selector with two naive strategies (all possible explanations and explanations only for the most likely prediction) and two baselines (no explanation and no AI support). The results suggest the potential of X-Selector to guide users to AI-suggested decisions and improve task performance under the condition of a high AI accuracy.

User Decision Guidance with Selective Explanation Presentation from Explainable-AI

TL;DR

This work tackles how to select explanations in explainable AI–driven decision support by introducing X-Selector, a method that predicts how different explanation combinations influence user decisions and selects the set that aligns with the AI's suggested decision. It leverages a UserModel to anticipate user responses and a policy π to characterize AI decisions, optimizing the match between human and AI choices via a discrepancy-minimizing objective. Through a stock-trading simulation with StockAI, the study demonstrates that selective explanations can outperform naive strategies, particularly when AI accuracy is high, while revealing limitations in low-accuracy contexts where AI unreliability reduces the benefit of guided explanations. These findings highlight the potential and limits of libertarian-paternalism-inspired explanation design in real-world IDSSs and point to future work on calibrating guidance strength and incorporating historical explanation context.

Abstract

This paper addresses the challenge of selecting explanations for XAI (Explainable AI)-based Intelligent Decision Support Systems (IDSSs). IDSSs have shown promise in improving user decisions through XAI-generated explanations along with AI predictions, and the development of XAI made it possible to generate a variety of such explanations. However, how IDSSs should select explanations to enhance user decision-making remains an open question. This paper proposes X-Selector, a method for selectively presenting XAI explanations. It enables IDSSs to strategically guide users to an AI-suggested decision by predicting the impact of different combinations of explanations on a user's decision and selecting the combination that is expected to minimize the discrepancy between an AI suggestion and a user decision. We compared the efficacy of X-Selector with two naive strategies (all possible explanations and explanations only for the most likely prediction) and two baselines (no explanation and no AI support). The results suggest the potential of X-Selector to guide users to AI-suggested decisions and improve task performance under the condition of a high AI accuracy.
Paper Structure (20 sections, 4 equations, 13 figures, 1 table)

This paper contains 20 sections, 4 equations, 13 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. XAIs for deep learning models
  4. Human-XAI interaction
  5. X-Selector
  6. Overview
  7. Algorithm
  8. Implementation
  9. Task
  10. StockAI
  11. Explanations
  12. Models
  13. Experiments
  14. Preliminary experiment
  15. Procedure
  16. ...and 5 more sections

Figures (13)

  • Figure 1: Chart
  • Figure 2: Examples of StockAI's prediction and its explanations
  • Figure 3: Screenshots of trading simulator
  • Figure 4: Structure of UserModel
  • Figure 5: High-accuracy
  • ...and 8 more figures