DUE: Dynamic Uncertainty-Aware Explanation Supervision via 3D Imputation

TL;DR

A Dynamic Uncertainty-aware Explanation supervision framework is proposed that ensures uncertainty-aware explanation guidance when dealing with sparsely annotated 3D data with diffusion-based 3D interpolation and is validated through comprehensive experiments on diverse real-world medical imaging datasets.

Abstract

Explanation supervision aims to enhance deep learning models by integrating additional signals to guide the generation of model explanations, showcasing notable improvements in both the predictability and explainability of the model. However, the application of explanation supervision to higher-dimensional data, such as 3D medical images, remains an under-explored domain. Challenges associated with supervising visual explanations in the presence of an additional dimension include: 1) spatial correlation changed, 2) lack of direct 3D annotations, and 3) uncertainty varies across different parts of the explanation. To address these challenges, we propose a Dynamic Uncertainty-aware Explanation supervision (DUE) framework for 3D explanation supervision that ensures uncertainty-aware explanation guidance when dealing with sparsely annotated 3D data with diffusion-based 3D interpolation. Our proposed framework is validated through comprehensive experiments on diverse real-world medical imaging datasets. The results demonstrate the effectiveness of our framework in enhancing the predictability and explainability of deep learning models in the context of medical imaging diagnosis applications.

Figures (6)

• Figure 1: Overview of the DUE framework: (a) presents the 3D explanation supervision, (b) demonstrates the Distance-Sensitive 3D interpolation, and (c) illustrates the Uncertain-Aware Explanation Guidance.
• Figure 2: Overview of the imputed uncertainty predictor training: A diffusion model is first trained for interpolation and uncertainty generation (solid orange line). Then, a Neural Processes (NP)-based VAE is trained to impute uncertainty for NP representations (dashed orange line). The red line represents the deployment path.
• Figure 3: Model performance under varying training sample sizes on the lung nodule classification dataset. (Left) Comparison of test prediction accuracy. (Middle) Comparison of test prediction ROC-AUC. (Right) Comparison of test IoU score.
• Figure 4: Visualizations display explanations for pancreatic tumor classification (left) and lung nodule classification (right). Human annotations are presented in the Mask columns, while model-generated explanations are depicted using heatmaps overlaid on the original images, highlighting regions of greater importance with warmer color intensities.
• Figure 5: IoU and F1 scores for model explanations at different thresholds on pancreatic tumor and lung nodule classification datasets.