Prototype Learning Guided Hybrid Network for Breast Tumor Segmentation in DCE-MRI

TL;DR

The paper tackles accurate breast tumor segmentation in DCE-MRI under computational constraints by introducing a prototype-guided hybrid network (PLHN) that fuses a lightweight 3D CNN encoder with a parallel transformer bottleneck and a prototype-based prediction module. Key innovations include two parallel encoders for efficient global and local feature modeling, online clustering to learn category prototypes, and an attention-based fusion that combines prototype similarity with decoder features to produce refined tumor masks. A two-stage optimization strategy, targeted patch sampling, and a robust loss design underpin effective training, while a radiomics-based CAD pipeline assesses HER2 status using automatically generated masks. Across internal and external datasets, PLHN achieves state-of-the-art segmentation performance with favorable computation costs and demonstrates potential for aiding clinical decision-making in breast cancer diagnosis and HER2 status prediction.

Abstract

Automated breast tumor segmentation on the basis of dynamic contrast-enhancement magnetic resonance imaging (DCE-MRI) has shown great promise in clinical practice, particularly for identifying the presence of breast disease. However, accurate segmentation of breast tumor is a challenging task, often necessitating the development of complex networks. To strike an optimal trade-off between computational costs and segmentation performance, we propose a hybrid network via the combination of convolution neural network (CNN) and transformer layers. Specifically, the hybrid network consists of a encoder-decoder architecture by stacking convolution and decovolution layers. Effective 3D transformer layers are then implemented after the encoder subnetworks, to capture global dependencies between the bottleneck features. To improve the efficiency of hybrid network, two parallel encoder subnetworks are designed for the decoder and the transformer layers, respectively. To further enhance the discriminative capability of hybrid network, a prototype learning guided prediction module is proposed, where the category-specified prototypical features are calculated through on-line clustering. All learned prototypical features are finally combined with the features from decoder for tumor mask prediction. The experimental results on private and public DCE-MRI datasets demonstrate that the proposed hybrid network achieves superior performance than the state-of-the-art (SOTA) methods, while maintaining balance between segmentation accuracy and computation cost. Moreover, we demonstrate that automatically generated tumor masks can be effectively applied to identify HER2-positive subtype from HER2-negative subtype with the similar accuracy to the analysis based on manual tumor segmentation. The source code is available at https://github.com/ZhouL-lab/PLHN.

Figures (7)

• Figure 1: An overview of the proposed prototype learning guided hybrid network. In the first stage of optimization, the concatenation of $[f_1^3,\tilde{f}_2]$ is fed into the decoder and the network is optimized on $P_1$. In the second stage of optimization and inference stage, the concatenation of $[f_1^3,f_t]$ is fed into the decoder and the network is optimized on $P_1$ and $P_f$ jointly.
• Figure 2: The overall pipeline of prototype guided breast tumor segmentation is illustrated by taking the normalized feature $X$ and the prototype $\mu$ as inputs. The dimension of feature is also listed.
• Figure 3: The visual comparison of segmentation results between different methods, such as DMFnet, MTLN, ResUnet, UXNET, MHL, Tumrosen, ALMN and PLHN, is displayed. Each row corresponds to one subject, and post-contrast images in axial plane overlaid with ground truth (red line) and automatic segmentation results (green line) of different methods are provided.
• Figure 4: The visual comparison of segmentation results between different methods for thymoma segmentation, such as Vnet, ResUnet, SwinUnet, UNETR, TransBTS, UXNET and the proposed PLHN, is displayed.
• Figure 5: Illustration of the similarity maps highlighted by different prototypes. Each heatmap corresponds to the similarity values between a particular prototype with image feature $X$. Brighter pixels indicate higher similarity values. (a) is the original image. (b) is the ground truth segmentation mask. (c) are the similarity maps highlighted by background prototypes. (d) are the similarity maps highlighted by foreground prototypes.