Shades of Uncertainty: How AI Uncertainty Visualizations Affect Trust in Alzheimer's Predictions

TL;DR

The paper tackles the problem of calibrating trust in AI-driven long-term Alzheimer's disease prognosis by examining how uncertainty visualizations influence user interpretation. It advances the field by treating uncertainty visualization as an explainable AI technique and comparing binary versus continuous encodings across general participants and domain experts, using a multimodal AD trajectory predictor (ML4VisAD) to generate color-coded prognostic visuals. The study finds that continuous uncertainty enhances perceived reliability among generalists and strengthens trust facets among experts, while binary uncertainty boosts momentary trust and confidence but may mask uncertainty and foster overconfidence. The authors provide six design guidelines to integrate uncertainty visuals with concise model context and multi-dimensional trust, emphasizing long-term prognosis and the need for external validation, with practical implications for building trustworthy, user-centered clinical decision support tools in AD prognosis.

Abstract

Artificial intelligence (AI) is increasingly used to support prognosis in Alzheimer's disease (AD), but adoption remains limited due to a lack of transparency and interpretability, particularly for long-term predictions where uncertainty is intrinsic and outcomes may not be known for years. We position uncertainty visualization as an explainable AI (XAI) technique and examine how it shapes trust, confidence, and reliance when users interpret AI-generated forecasts of future cognitive decline transitions. We conducted two studies, one with general participants (N=37) and one with experts in neuroimaging and neurology (N=10), to compare binary (present/absent) and continuous (saturation) uncertainty encodings. Continuous encodings improved perceived reliability and helped users recognize model limitations, while binary encodings increased momentary confidence, revealing expertise-dependent trade-offs in interpreting future predictions under high uncertainty. These findings surface key challenges in designing uncertainty representations for prognostic AI and culminate in a set of empirically grounded guidelines for creating trustworthy, user-appropriate clinical decision support tools.

Figures (9)

• Figure 1: Example stimuli. Each row shows a case (stable CN, converting MCI$\rightarrow$AD, stable MCI). Columns show binary (left) vs. continuous (right) uncertainty formats.
• Figure 2: Information elements shown to participants in the "moderate information’’ condition, adapted from Lai et al.lai2023towards.
• Figure 3: Effect of model information levels on trust.
• Figure 4: Comparison of binary versus continuous uncertainty encodings and their effects on trust.
• Figure 5: Comparison of visual representations of AI uncertainty (binary vs continuous) using the multi-dimensional scheme of trust according to Ashoori and Weisz ashoori2019ai.