DFEN: Dual Feature Equalization Network for Medical Image Segmentation

TL;DR

DFEN addresses the problem of misclassification caused by unequal contextual information near boundaries and for small-class regions in medical image segmentation. It introduces image-level feature equalization (ILFEM) and class-level feature equalization (CLFEM) within a Swin Transformer–CNN framework, augmented by additive and concatenative upsampling to enhance pixel representations. The model is trained with a combined Dice and cross-entropy loss $L_{loss} = \alpha L_{CE}(p,y) + \beta L_{Dice}(p,y)$, with empirically tuned weights (e.g., $\alpha=0.3$, $\beta=0.7$), and demonstrates state-of-the-art performance across four datasets (ACDC, ISIC2017, PH$^2$, BUSI), along with comprehensive ablations showing the gains from ILFEM and CLFEM. These contributions advance accurate boundary delineation and small-region segmentation, with potential to improve clinical decision support while maintaining relatively lightweight parameters.

Abstract

Current methods for medical image segmentation primarily focus on extracting contextual feature information from the perspective of the whole image. While these methods have shown effective performance, none of them take into account the fact that pixels at the boundary and regions with a low number of class pixels capture more contextual feature information from other classes, leading to misclassification of pixels by unequal contextual feature information. In this paper, we propose a dual feature equalization network based on the hybrid architecture of Swin Transformer and Convolutional Neural Network, aiming to augment the pixel feature representations by image-level equalization feature information and class-level equalization feature information. Firstly, the image-level feature equalization module is designed to equalize the contextual information of pixels within the image. Secondly, we aggregate regions of the same class to equalize the pixel feature representations of the corresponding class by class-level feature equalization module. Finally, the pixel feature representations are enhanced by learning weights for image-level equalization feature information and class-level equalization feature information. In addition, Swin Transformer is utilized as both the encoder and decoder, thereby bolstering the ability of the model to capture long-range dependencies and spatial correlations. We conducted extensive experiments on Breast Ultrasound Images (BUSI), International Skin Imaging Collaboration (ISIC2017), Automated Cardiac Diagnosis Challenge (ACDC) and PH$^2$ datasets. The experimental results demonstrate that our method have achieved state-of-the-art performance. Our code is publicly available at https://github.com/JianJianYin/DFEN.

Figures (7)

• Figure 1: Visualization results on the BUSI dataset compared with the other state-of-the-art method UNeXtvalanarasu2022unext. The black area show the background and the white area show the tumor. The red area indicates the area that was misclassified for each method. DSC represents the Dice Similarity Score for the entire image. GT stands for ground truth label.
• Figure 2: The network structure of the DFEN model. DFEN is a hybrid framework based on Swin Transformer and CNN. Except for the dual feature equalization module, which is based on CNN, all other modules are based on Swin Transformer. The dual feature equalization module (ILFEM and CLFEM) is dedicated to enhancing pixel feature representations by utilizing class-level equalization feature information and image-level equalization feature information. Add-Upsampling refers to additive upsampling, which is used to obtain several additive upsampling features ($F_1$, $F_2$, $F_3$) by fusing the features generated by the encoder ($R_{0}$, $R_1$, $R$) and upsampling of the corresponding depth. Similarly, Con-Upsampling is a concatenative upsampling that focuses on upsampling these additive upsampling features, and finally concatenating the channel dimensions of the features. $Z$ is the number of classes.
• Figure 3: The internal structure diagram of the Swin Transformer Block. The Swin Transformer mainly consists of layer normalization(LN), window-based multi-head self-attention(W-MSA), shifted window-based multi-head self-attention(SW-MSA), and multi-layer perceptron(MLP).
• Figure 4: The overview diagram of the image-level feature equalization module. $R_{ci}$ is the coarse image-level equalization feature representations. and $R_{il}$ is the fine-grained image-level equalization feature representations.
• Figure 5: The overview diagram of the class-level feature equalization module. $R_{cc}$ is the coarse class-level equalization feature representations, and $R_{cl}$ is the fine-grained class-level equalization feature representations.