To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events

TL;DR

The paper advances understanding of AI-enabled decision support in time-critical medical events by contrasting information-driven AI outputs against recommendation-driven outputs. Through design research at a pediatric trauma center and an online experiment with 35 providers across six systems, it demonstrates that AI information and recommendations significantly improve decision accuracy for life-saving interventions, while information synthesis alone may not. It uncovers socio-technical barriers, including an accuracy-time trade-off and polarized perceptions of recommendations, and offers three practical implications: support critical evaluation of synthesized information, expand information-driven strategies beyond prescriptions, and establish clear responsibility policies for AI usage. The work contributes to human-AI interaction in high-stakes, dynamic environments and provides guidance for deploying AI-enabled decision support that preserves clinician agency and team coordination.

Abstract

AI-enabled decision-support systems aim to help medical providers rapidly make decisions with limited information during medical emergencies. A critical challenge in developing these systems is supporting providers in interpreting the system output to make optimal treatment decisions. In this study, we designed and evaluated an AI-enabled decision-support system to aid providers in treating patients with traumatic injuries. We first conducted user research with physicians to identify and design information types and AI outputs for a decision-support display. We then conducted an online experiment with 35 medical providers from six health systems to evaluate two human-AI interaction strategies: (1) AI information synthesis and (2) AI information and recommendations. We found that providers were more likely to make correct decisions when AI information and recommendations were provided compared to receiving no AI support. We also identified two socio-technical barriers to providing AI recommendations during time-critical medical events: (1) an accuracy-time trade-off in providing recommendations and (2) polarizing perceptions of recommendations between providers. We discuss three implications for developing AI-enabled decision support used in time-critical events, contributing to the limited research on human-AI interaction in this context.

Figures (9)

• Figure 1: Examples of recommendation and information display mockups used to elicit participant feedback in the interviews. The top row has mockups of an information-synthesis display (A.1 and B.1) while the bottom row has mockups of an information-and-recommendation display (A.2 and B.2).
• Figure 2: An overview of the online experiment flow. Participants experienced three conditions: (1) No AI Support, (2) Display with AI Information Synthesis, and (3) Display with AI Information and Recommendations. Participants were randomly split into groups, with the order of conditions counterbalanced across the groups. In the two conditions with a display, information was dynamically synthesized on the display as it was presented in the voiceover. In the condition with recommendations, recommendations and risk predictions for both blood transfusion and neurosurgical intervention appeared at the end.
• Figure 3: An example of a vignette included in the online experiment. The vignette contained a countdown clock and an embedded video with a voiceover of the case. In vignettes with AI Information Synthesis or AI Information and Recommendations, the video also included the AI support display. The questions about the patient's need for LSIs were located below the video.
• Figure 4: Diagnostic accuracy of decision-making instances across the three types of AI support.
• Figure 5: Timing of initial answers to the need for blood transfusion question. Information was presented in the same order for each vignette. The bar graph shows the number of new answers selected after a piece of information was presented (e.g., 41 new answers were selected after the mechanism of injury was presented). The bar graph also highlights the proportion of correct (gray) and incorrect (red) new answers. The line graph indicates the number of total initial answers (e.g., 44 total answers had been made from the start of the video to after presentation of mechanism of injury). An asterisk (*) indicates that the piece of information was also included on the decision-support display.