A User Study Evaluating Argumentative Explanations in Diagnostic Decision Support

TL;DR

This study empirically evaluates how argumentative explanations derived from XAI outputs affect clinicians' understanding and trust in AI-assisted diagnostic decision support for transient loss of consciousness. By converting LIME feature attributions and counterfactuals into three templates (attribution, counterfactual, exclusion), the authors create verbal arguments and assess them with eight neurologists across ten real cases. Results show attribution-based explanations are most comprehensible and plausible, yet clinicians rarely endorse using AI-provided explanations to discuss cases with colleagues, due to missing information about the quality and uncertainty of the AI predictions and a lack of case-specific detail. The work highlights the need for reliability signals and adaptable, interactive explanations to improve acceptability and trust in clinical AI tools, and suggests future work to enhance specificity and combine multiple XAI methods for richer argumentative explanations.

Abstract

As the field of healthcare increasingly adopts artificial intelligence, it becomes important to understand which types of explanations increase transparency and empower users to develop confidence and trust in the predictions made by machine learning (ML) systems. In shared decision-making scenarios where doctors cooperate with ML systems to reach an appropriate decision, establishing mutual trust is crucial. In this paper, we explore different approaches to generating explanations in eXplainable AI (XAI) and make their underlying arguments explicit so that they can be evaluated by medical experts. In particular, we present the findings of a user study conducted with physicians to investigate their perceptions of various types of AI-generated explanations in the context of diagnostic decision support. The study aims to identify the most effective and useful explanations that enhance the diagnostic process. In the study, medical doctors filled out a survey to assess different types of explanations. Further, an interview was carried out post-survey to gain qualitative insights on the requirements of explanations incorporated in diagnostic decision support. Overall, the insights gained from this study contribute to understanding the types of explanations that are most effective.

Figures (6)

• Figure 1: Three-layer Bayesian Network with the 36 most relevant variables determined by Wardrope et al. Wardrope2020_MachineLearningDiagnosticDecision.
• Figure 2: User ratings on the comprehensibility of the three generated argument types.
• Figure 3: User ratings on the plausibility of the three generated argument types.
• Figure 4: User ratings on completeness of the three generated argument types.
• Figure 5: User ratings along all dimensions combined for each argument type.