The Limits of Perception: Analyzing Inconsistencies in Saliency Maps in XAI
Anna Stubbin, Thompson Chyrikov, Jim Zhao, Christina Chajo
TL;DR
The paper tackles inconsistencies in saliency-map explanations within medical XAI for imaging by proposing a comprehensive framework that fuses domain-specific knowledge, robust modeling, adversarial training, and calibrated post-hoc explanations with clinician validation. It augments saliency analysis with counterfactual explanations and continuous feedback, and defines an evaluation regime focusing on explanation fidelity, consistency, and robustness. Empirical results indicate improved feature-importance ranking, reduced explanation variance under perturbations, and higher clinician consensus, supporting safer and more trustworthy AI-assisted diagnostics. The work underscores the importance of transparency and open collaboration to integrate AI tools into healthcare effectively, with implications for widespread adoption and patient outcomes.
Abstract
Explainable artificial intelligence (XAI) plays an indispensable role in demystifying the decision-making processes of AI, especially within the healthcare industry. Clinicians rely heavily on detailed reasoning when making a diagnosis, often CT scans for specific features that distinguish between benign and malignant lesions. A comprehensive diagnostic approach includes an evaluation of imaging results, patient observations, and clinical tests. The surge in deploying deep learning models as support systems in medical diagnostics has been significant, offering advances that traditional methods could not. However, the complexity and opacity of these models present a double-edged sword. As they operate as "black boxes," with their reasoning obscured and inaccessible, there's an increased risk of misdiagnosis, which can lead to patient harm. Hence, there is a pressing need to cultivate transparency within AI systems, ensuring that the rationale behind an AI's diagnostic recommendations is clear and understandable to medical practitioners. This shift towards transparency is not just beneficial -- it's a critical step towards responsible AI integration in healthcare, ensuring that AI aids rather than hinders medical professionals in their crucial work.