A Dempster-Shafer approach to trustworthy AI with application to fetal brain MRI segmentation

TL;DR

The paper tackles trustworthiness in medical image segmentation by introducing a Dempster-Shafer theory–based framework that couples a high-accuracy backbone with a robust atlas-based fallback and a voxelwise fail-safe. By modeling contracts of trust as BPAs and using DS combination, it detects conflicts and locally switches to the fallback to discard anatomically implausible predictions, improving robustness on diverse fetal brain MRI data. Evaluated on 540 multi-center 3D MRIs, the approach yields improved Dice and HD metrics and higher radiologist trust, demonstrating the practical value of integrating anatomical and intensity priors with a principled safety mechanism. The work also clarifies the context-dependent nature of trust and outlines avenues for extension to other segmentation tasks and regulatory reporting.

Abstract

Deep learning models for medical image segmentation can fail unexpectedly and spectacularly for pathological cases and images acquired at different centers than training images, with labeling errors that violate expert knowledge. Such errors undermine the trustworthiness of deep learning models for medical image segmentation. Mechanisms for detecting and correcting such failures are essential for safely translating this technology into clinics and are likely to be a requirement of future regulations on artificial intelligence (AI). In this work, we propose a trustworthy AI theoretical framework and a practical system that can augment any backbone AI system using a fallback method and a fail-safe mechanism based on Dempster-Shafer theory. Our approach relies on an actionable definition of trustworthy AI. Our method automatically discards the voxel-level labeling predicted by the backbone AI that violate expert knowledge and relies on a fallback for those voxels. We demonstrate the effectiveness of the proposed trustworthy AI approach on the largest reported annotated dataset of fetal MRI consisting of 540 manually annotated fetal brain 3D T2w MRIs from 13 centers. Our trustworthy AI method improves the robustness of a state-of-the-art backbone AI for fetal brain MRIs acquired across various centers and for fetuses with various brain abnormalities.

Figures (11)

• Figure 1: Schematics of our principled method for trustworthy AI applied to medical image segmentation.a. Deep neural networks for medical image segmentation (AI algorithm) are typically trained on images from a limited number of acquisition centers. This is usually not sufficient to cover all the anatomical variability. b. When such a trained AI algorithm is deployed, it will typically give satisfactory accuracy for images acquired with the same protocol as training images and with a health condition represented in the training dataset (left). However, an AI algorithm might fail with errors that are not anatomically plausible, for images acquired with a slightly different protocol as training images and/or representing anatomy underrepresented in the training dataset (right). c. Schematic of the proposed trustworthy AI algorithm. A backbone AI segmentation algorithm is coupled with a fallback segmentation algorithm. Experts knowledge about the anatomy is modeled in the fail-safe mechanism using Dempster-Shafer theory. A rich variety of experts knowledge can be modeled as contracts of trust, such as, but not only, atlas-based prior and intensity-based prior (not shown here). When part of the AI segmentation is found to contradict one of the contracts of trust for a voxel, our trustworthy AI algorithm automatically switches continuously to the fallback segmentation for this voxel.
• Figure 2: Illustration of the improved robustness of the proposed trustworthy AI method (TW-AI) as compared to nnU-Net state-of-the-art backbone AI method. (Top) 3D MRI of a neurotypical fetus at 28 weeks of gestation acquired at the same center as the training data for the AI. (Bottom) 3D MRI of a fetus with a high-flow dural sinus malformation at 28 weeks of gestation acquired at a different center as the training data for the AI. Severe violations of the anatomy by the backbone AI are highlighted. The TW-AI does not make those errors.
• Figure 3: Mean-ROI Trustworthiness Scores for out-of-scanner distribution 3D MRIs. We report four scoring by a panel of eight experts of the trustworthiness of the automatic segmentations for a subset of the out-of-scanner distribution testing 3D MRIs ($n=50$). Each expert was asked to score from $0$ (totally unacceptable) to 5 (perfect fit) the trustworthiness of each ROI. The scores displayed here are averaged across ROIs. AI corresponds to nnU-Net isensee2021nnu here. Results per ROI can be found in the appendix (Fig. \ref{['fig:scores_roi']}).
• Figure 4: Comparison of the backbone AI, fallback, and trustworthy AI segmentation algorithms across gestational ages, for neurotypical and spina bifida cases. AI corresponds to nnU-Net isensee2021nnu here. Results per ROI can be found in the appendix (Fig. \ref{['fig:dice_GA_roi']}, \ref{['fig:hausdorff_GA_roi']}).
• Figure A.1: Composition of the training and testing datasets (total: 540 3D MRIs)In-scanner distribution designates the 3D MRIs acquired at the same center as the training data. Out-of-scanner distribution designates the 3D MRIs acquired at different centers than the training data. This is the largest fetal brain MRI dataset reported to date.