LCE: A Framework for Explainability of DNNs for Ultrasound Image Based on Concept Discovery

TL;DR

The paper tackles the lack of interpretable deep learning for ultrasound breast imaging by introducing the Lesion Concept Explainer (LCE), which fuses attribution-based guidance with Segment Anything Model (SAM) concept discovery and uses Shapley values to derive faithful explanations. A new evaluation metric, Effect Score, combines faithfulness with explanation size, and LCE is validated on public BUSI and private FG-US-B datasets, showing superior faithfulness and understandability versus baselines. The approach is model-agnostic and resource-efficient, leveraging a medically fine-tuned SAM (SAMMed2D) to discover lesion concepts without heavy annotation. The work advances trustworthy AI for low-cost ultrasound imaging and supports robust, fine-grained diagnostic explanations in clinical contexts.

Abstract

Explaining the decisions of Deep Neural Networks (DNNs) for medical images has become increasingly important. Existing attribution methods have difficulty explaining the meaning of pixels while existing concept-based methods are limited by additional annotations or specific model structures that are difficult to apply to ultrasound images. In this paper, we propose the Lesion Concept Explainer (LCE) framework, which combines attribution methods with concept-based methods. We introduce the Segment Anything Model (SAM), fine-tuned on a large number of medical images, for concept discovery to enable a meaningful explanation of ultrasound image DNNs. The proposed framework is evaluated in terms of both faithfulness and understandability. We point out deficiencies in the popular faithfulness evaluation metrics and propose a new evaluation metric. Our evaluation of public and private breast ultrasound datasets (BUSI and FG-US-B) shows that LCE performs well compared to commonly-used explainability methods. Finally, we also validate that LCE can consistently provide reliable explanations for more meaningful fine-grained diagnostic tasks in breast ultrasound.

Figures (5)

• Figure 1: The Lesion Concept Explainer (LCE) Framework. For any black-box DNN model, attribution methods are first used to generate explanations, which are then used to guide $D_{SAM}$ to explore and obtain concept masks. The concept masks are then post-processed. Finally, the Shapley value is applied to identify the explanations that are most crucial for model's decisions.
• Figure 2: The Shapley value calculated when a black image is diagnosed as benign with a 96% probability is not faithful.
• Figure 3: Examples of datasets.
• Figure 4: The $Insertion$ curve responds to the extent to which the confidence of the model predictions correlates with the original predictions as the percentage of explanation inserted increases. The EAC generates some meaningless explanations, but those explanations receive high evaluation results (first row). In contrast, LIME generates explanations that do not match certain concepts, but are not as meaningless as EAC (second row). $\mathrm{E}_i$ indicates that this is the $i$-th explanation accumulation image produced in the computation of $Insertion$.
• Figure 5: Sample explanations generated by LCE, EACsun2024explain and six baseline methodsribeiro2016shouldsundararajan2017axiomaticlundberg2017unifiedshrikumar2017learningselvaraju2017grad.