Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Explanatory Model Steering System for Collaboration between Domain Experts and AI

Aditya Bhattacharya, Simone Stumpf, Katrien Verbert

TL;DR

The paper tackles the challenge of integrating domain experts into AI development for high-stakes domains by proposing an Explanatory Model Steering system. It combines an explanation dashboard with data-centric and model-centric explanations and supports manual and automated data configuration to steer training data and thus models, implemented as a three-layer architecture (UI, middleware, XAI engine) and containerized; it is evaluated in healthcare with 174 experts across three studies, with code available publicly. The authors detail the system design, the multifaceted explanation dashboard, and the data configuration methods, and report findings that expert involvement and diverse explanations enhance human-AI collaboration and steering effectiveness. The work offers a practical pathway to improve trust, understanding, and control in AI-assisted decision making and is poised for generalization to domains beyond healthcare.

Abstract

With the increasing adoption of Artificial Intelligence (AI) systems in high-stake domains, such as healthcare, effective collaboration between domain experts and AI is imperative. To facilitate effective collaboration between domain experts and AI systems, we introduce an Explanatory Model Steering system that allows domain experts to steer prediction models using their domain knowledge. The system includes an explanation dashboard that combines different types of data-centric and model-centric explanations and allows prediction models to be steered through manual and automated data configuration approaches. It allows domain experts to apply their prior knowledge for configuring the underlying training data and refining prediction models. Additionally, our model steering system has been evaluated for a healthcare-focused scenario with 174 healthcare experts through three extensive user studies. Our findings highlight the importance of involving domain experts during model steering, ultimately leading to improved human-AI collaboration.

An Explanatory Model Steering System for Collaboration between Domain Experts and AI

TL;DR

The paper tackles the challenge of integrating domain experts into AI development for high-stakes domains by proposing an Explanatory Model Steering system. It combines an explanation dashboard with data-centric and model-centric explanations and supports manual and automated data configuration to steer training data and thus models, implemented as a three-layer architecture (UI, middleware, XAI engine) and containerized; it is evaluated in healthcare with 174 experts across three studies, with code available publicly. The authors detail the system design, the multifaceted explanation dashboard, and the data configuration methods, and report findings that expert involvement and diverse explanations enhance human-AI collaboration and steering effectiveness. The work offers a practical pathway to improve trust, understanding, and control in AI-assisted decision making and is poised for generalization to domains beyond healthcare.

Abstract

With the increasing adoption of Artificial Intelligence (AI) systems in high-stake domains, such as healthcare, effective collaboration between domain experts and AI is imperative. To facilitate effective collaboration between domain experts and AI systems, we introduce an Explanatory Model Steering system that allows domain experts to steer prediction models using their domain knowledge. The system includes an explanation dashboard that combines different types of data-centric and model-centric explanations and allows prediction models to be steered through manual and automated data configuration approaches. It allows domain experts to apply their prior knowledge for configuring the underlying training data and refining prediction models. Additionally, our model steering system has been evaluated for a healthcare-focused scenario with 174 healthcare experts through three extensive user studies. Our findings highlight the importance of involving domain experts during model steering, ultimately leading to improved human-AI collaboration.
Paper Structure (7 sections, 3 figures)

This paper contains 7 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Explanatory Model Steering System
  3. Explanation Dashboard
  4. Data Configuration Approaches
  5. Technical Implementation
  6. Use Case: Steering Diabetes Prediction System
  7. Conclusion

Figures (3)

  • Figure 1: Screenshot of the explanation dashboard which combines diverse data-centric and model-centric global explanations. The different visual components are designed based on the guidelines from Bhattacharya et al. bhattacharya2024exmos.
  • Figure 2: Screenshots of the manual and automated configuration screens from the system.
  • Figure 3: The model steering application adopts a three-layered architecture to facilitate user interactions.