Enhancing breast cancer detection on screening mammogram using self-supervised learning and a hybrid deep model of Swin Transformer and Convolutional Neural Network

TL;DR

This work tackles the challenge of limited labeled mammography data by employing self-supervised pretraining with EsViT to initialize a Swin-T backbone, followed by HybMNet that fuses global transformer features with local CNN details from ROI patches. The two-stage approach achieves strong performance on CMMD and INbreast, with SSL pretraining yielding a notable AUC boost (e.g., up to $0.864$ on CMMD and $0.889$ on INbreast) and improved F1 scores. By integrating ROI-based fine-grained analysis with global context through a fusion mechanism, the method demonstrates robustness and potential as a second reader in clinical workflows. The findings highlight the value of domain-specific SSL pretraining for high-resolution medical imaging and point toward better generalization with more diverse mammography data and augmentations.

Abstract

Purpose: The scarcity of high-quality curated labeled medical training data remains one of the major limitations in applying artificial intelligence (AI) systems to breast cancer diagnosis. Deep models for mammogram analysis and mass (or micro-calcification) detection require training with a large volume of labeled images, which are often expensive and time-consuming to collect. To reduce this challenge, we proposed a novel method that leverages self-supervised learning (SSL) and a deep hybrid model, named \textbf{HybMNet}, which combines local self-attention and fine-grained feature extraction to enhance breast cancer detection on screening mammograms. Approach: Our method employs a two-stage learning process: (1) SSL Pretraining: We utilize EsViT, a SSL technique, to pretrain a Swin Transformer (Swin-T) using a limited set of mammograms. The pretrained Swin-T then serves as the backbone for the downstream task. (2) Downstream Training: The proposed HybMNet combines the Swin-T backbone with a CNN-based network and a novel fusion strategy. The Swin-T employs local self-attention to identify informative patch regions from the high-resolution mammogram, while the CNN-based network extracts fine-grained local features from the selected patches. A fusion module then integrates global and local information from both networks to generate robust predictions. The HybMNet is trained end-to-end, with the loss function combining the outputs of the Swin-T and CNN modules to optimize feature extraction and classification performance. Results: The proposed method was evaluated for its ability to detect breast cancer by distinguishing between benign (normal) and malignant mammograms. Leveraging SSL pretraining and the HybMNet model, it achieved AUC of 0.864 (95% CI: 0.852, 0.875) on the CMMD dataset and 0.889 (95% CI: 0.875, 0.903) on the INbreast dataset, highlighting its effectiveness.

Figures (5)

• Figure 1: Overview of the proposed breast cancer detection method, consisting of two parts: (1) SSL pretraining of the Swin-T backbone using EsViT on full-resolution mammograms. The multi-stage Swi-T processes each mammogram by organizing it into a sequence of smaller patches. Sparse self-attention mechanisms are applied to capture fine-grained features while reducing computational complexity. At intermediate layers, neighboring tokens are merged hierarchically to further improve efficiency. The pretrained Swin-T model with the lowest loss is then used to initialize the feature extractor for the downstream HybMNet. (2) Downstream training of HybMNet that integrates a Swin-T branch and a CNN branch. The Swin-T branch utilizes the pretrained Swin-T to generate a saliency map, identifying the most informative regions (ROIs) in the input mammogram. These candidate ROI patches are cropped from the original image and passed to the CNN branch for extracting detailed local features. A fusion module combines global features from the Swin-T branch with local features from the CNN branch to produce robust overall predictions for breast cancer detection.
• Figure 2: Comparison with state-of-the-art methods on CMMD test set. Error bars represent 95% CIs and the centers correspond to the mean of each classification metric across different models.
• Figure 3: Comparison with state-of-the-art methods on INbreast test set. Error bars represent 95% CIs and the centers correspond to the mean of each classification metric across different models.
• Figure 4: Ablation study results on CMMD test set. Error bars represent 95% CIs and the centers correspond to the mean of each classification metric across different models.
• Figure 5: Visualization of ROI patches. From left to right, the input mammograms annotated with calcification segmentation labels, the identified ROI patch locations (highlighted in blue), and three selected ROI patches containing calcifications (marked in red).