The case for delegated AI autonomy for Human AI teaming in healthcare

TL;DR

An advanced approach to integrating artificial intelligence into healthcare: autonomous decision support that allows the AI algorithm to act autonomously for a subset of patient cases whilst serving a supportive role in other subsets of patient cases based on defined delegation criteria is proposed.

Abstract

In this paper we propose an advanced approach to integrating artificial intelligence (AI) into healthcare: autonomous decision support. This approach allows the AI algorithm to act autonomously for a subset of patient cases whilst serving a supportive role in other subsets of patient cases based on defined delegation criteria. By leveraging the complementary strengths of both humans and AI, it aims to deliver greater overall performance than existing human-AI teaming models. It ensures safe handling of patient cases and potentially reduces clinician review time, whilst being mindful of AI tool limitations. After setting the approach within the context of current human-AI teaming models, we outline the delegation criteria and apply them to a specific AI-based tool used in histopathology. The potential impact of the approach and the regulatory requirements for its successful implementation are then discussed.

Figures (3)

• Figure 1: Current Human-AI teaming modalities. This figure shows two clinical workflow models for AI-based decision support tools which represents the current approach to integrating AI into clinical decision making processes. The panel (a) represents a sequential clinical workflow, where input data is first processed by clinicians and then passed to the AI-based tool in a linear, step-by-step manner before arriving at the final decision made by the clinician. The panel (b) shows a concurrent clinical workflow, where clinicians and the AI-based tool simultaneously receive and process input data, then the clinician makes the final decision.
• Figure 2: The clinical workflow for an autonomous decision support tool. This figure shows our approach for integrating AI in healthcare, where it has a delegation criteria, which interacts with the AI-based tool, to direct the input patient data to three distinctive pathways, "AI only", "Clinician only", and "Clinician and AI together". "AI only" pathway represents the "autonomous" element in our approach and "Clinician and AI together" pathway represents the "support" element in our approach, hence the name "autonomous decision support" approach.
• Figure 3: The potential clinical workflow for using COBIx as an autonomous decision support tool. COBIx is a published multi-class AI-based tool for analysing colon and rectal (large bowel) endoscopic biopsies. It produces five possible outputs with one normal category and four abnormal categories, i.e. Neoplastic urgent, Neoplastic non-urgent, Non-neoplastic urgent, Non-neoplastic non-urgent. This workflow shows how Clinical context, AI failure mode, Criticality of decision, and AI confidence score can be incorporated into designing the delegation criteria to safely direct the biopsies into three pathways: "AI only", "Clinician only" and "Clinician and AI together".