Supporting Data-Frame Dynamics in AI-assisted Decision Making

TL;DR

The paper addresses high-stakes decision making where evidence evolves over time, arguing that current AI decision aids fixate on single frames rather than supporting data-frame dynamics. It proposes a mixed-initiative workflow grounded in data-frame theory, formalizing core processes with $I$, $E_t$, $\mathcal{H}_t$, and functions $f_t$, $r_t$, $s$, plus a threshold $\delta$ to determine hypothesis acceptance. A skin cancer diagnosis prototype demonstrates interactive evidence extraction, hypothesis retrieval, and scoring, leveraging a concept bottleneck model and Grad-CAM to visualize evidence, with conformal prediction guiding hypothesis suggestions. The approach aims to reduce cognitive load, prevent overreliance, and enable ongoing human–AI collaboration to improve decision quality in high-stakes domains, with plans for clinical evaluation in dermatology.

Abstract

High stakes decision-making often requires a continuous interplay between evolving evidence and shifting hypotheses, a dynamic that is not well supported by current AI decision support systems. In this paper, we introduce a mixed-initiative framework for AI assisted decision making that is grounded in the data-frame theory of sensemaking and the evaluative AI paradigm. Our approach enables both humans and AI to collaboratively construct, validate, and adapt hypotheses. We demonstrate our framework with an AI-assisted skin cancer diagnosis prototype that leverages a concept bottleneck model to facilitate interpretable interactions and dynamic updates to diagnostic hypotheses.

Figures (2)

• Figure 1: (A) Typical AI decision support: The human’s decision and the AI’s prediction can be at odds, requiring the human to interpret the AI’s output and resolve conflicts independently; (B) The evaluative AI framework: The AI generates hypotheses and corresponding evidence to facilitate sensemaking. However, how to generate the support to well synchronize with human reasoning is unclear; (C) Instantiate evaluative AI with mixed-initiative sensemaking: Humans can propose reasoning, and the AI actively integrates these updates to offer support, enabling collaborative, team-based decision making.
• Figure 2: An example system that implements the mixed-initiative sensemaking support for skin cancer diagnosis (the image is from the derm7pt dataset kawahara2018seven).