Morphology-Enhanced CAM-Guided SAM for weakly supervised Breast Lesion Segmentation

TL;DR

This study tackles the challenge of breast lesion segmentation in ultrasound with image-level supervision by integrating morphology-based contour extraction, CAM-driven semantic localization, and SAM-based refinement. The four-stage framework fuses traditional morphological cues with deep semantic cues and leverages a prompt-driven SAM to achieve detailed segmentation, using LayerCAM for localization and box prompts for guidance. On BUSI and Dataset B, the method achieves a Dice score of $74.39\%$ and a Hausdorff distance HD95 of $24.27$ on BUSI, approaching the performance of fully supervised methods while outperforming other weakly supervised baselines, thereby reducing annotation costs. The approach demonstrates strong boundary delineation and robustness, with potential applicability to broader medical imaging tasks and modalities.

Abstract

Ultrasound imaging plays a critical role in the early detection of breast cancer. Accurate identification and segmentation of lesions are essential steps in clinical practice, requiring methods to assist physicians in lesion segmentation. However, ultrasound lesion segmentation models based on supervised learning require extensive manual labeling, which is both time-consuming and labor-intensive. In this study, we present a novel framework for weakly supervised lesion segmentation in early breast ultrasound images. Our method uses morphological enhancement and class activation map (CAM)-guided localization. Finally, we employ the Segment Anything Model (SAM), a computer vision foundation model, for detailed segmentation. This approach does not require pixel-level annotation, thereby reducing the cost of data annotation. The performance of our method is comparable to supervised learning methods that require manual annotations, achieving a Dice score of 74.39% and outperforming comparative supervised models in terms of Hausdorff distance in the BUSI dataset. These results demonstrate that our framework effectively integrates weakly supervised learning with SAM, providing a promising solution for breast cancer image analysis. The code for this study is available at: https://github.com/YueXin18/MorSeg-CAM-SAM.

Figures (6)

• Figure 1: The proposed framework of our model consists of four key stages: in stage 1, we perform a preliminary image segmentation focusing on morphological features. This is followed by the generation of a heat map for lesion localization using a CAM-based classification network, as shown in stage 2. Subsequently, the features from the previous stages were synthesized as shown in stage 3 and generated as a bounding box prompt in stage 4 for detailed segmentation using the SAM.
• Figure 2: The feature fusion and lesion synthesis process. This step extracts the contours of each lesion in the traditional morphology-based segmentation results and determines the lesion with the largest area of intersection with the results of the CAM-guided lesion localization module as the synthesized result.
• Figure 3: Visualization of different Class Activation Mapping (CAM) Methods at a threshold value of 200 on the BUSI dataset.
• Figure 4: Visualization results from ablation study on the BUSI dataset.
• Figure 5: Visualization results of different versions of SAM segmentation on the BUSI dataset.