Selective Complementary Feature Fusion and Modal Feature Compression Interaction for Brain Tumor Segmentation

TL;DR

This work tackles brain glioma segmentation from multi-modal MRI by addressing cross-modal feature disparity and bottom-end feature dimensionality growth in parallel architectures. It introduces CFCI-Net, combining Selective Complementary Feature Fusion (SCFF) to adaptively fuse cross-modal information with complementary weights, and a Modal Feature Compression Interaction (MFCI) Transformer to compress features and enable robust multi-modal interactions. Extensive experiments on BraTS2019 and BraTS2020 demonstrate state-of-the-art Dice scores across ET, WT, and TC with competitive Hausdorff distances, supported by comprehensive ablations that validate the contributions of SCFF and MFCI. The approach offers a principled, efficient pathway for leveraging complementary modal information in multi-modal MRI segmentation with potential impact on clinical decision-support pipelines.

Abstract

Efficient modal feature fusion strategy is the key to achieve accurate segmentation of brain glioma. However, due to the specificity of different MRI modes, it is difficult to carry out cross-modal fusion with large differences in modal features, resulting in the model ignoring rich feature information. On the other hand, the problem of multi-modal feature redundancy interaction occurs in parallel networks due to the proliferation of feature dimensions, further increase the difficulty of multi-modal feature fusion at the bottom end. In order to solve the above problems, we propose a noval complementary feature compression interaction network (CFCI-Net), which realizes the complementary fusion and compression interaction of multi-modal feature information with an efficient mode fusion strategy. Firstly, we propose a selective complementary feature fusion (SCFF) module, which adaptively fuses rich cross-modal feature information by complementary soft selection weights. Secondly, a modal feature compression interaction (MFCI) transformer is proposed to deal with the multi-mode fusion redundancy problem when the feature dimension surges. The MFCI transformer is composed of modal feature compression (MFC) and modal feature interaction (MFI) to realize redundancy feature compression and multi-mode feature interactive learning. %In MFI, we propose a hierarchical interactive attention mechanism based on multi-head attention. Evaluations on the BraTS2019 and BraTS2020 datasets demonstrate that CFCI-Net achieves superior results compared to state-of-the-art models. Code: https://github.com/CDmm0/CFCI-Net

Figures (6)

• Figure 1: The architecture of the proposed CFCI-Net. The whole model consists of four parallel encoders, MFCI transformer, SCFF module and decoder. The encoder part sets the mode feature extraction path for each mode and is composed of ResNet module. The decoder consists of convolution and upsampling interpolation.
• Figure 2: The architecture of the SCFF, it consists of two identical modal feature extraction paths. According to the characteristics of different modes, T1 and T2 are set as a group, and T1ce and Flair are set as a group.
• Figure 3: The architecture of the modal feature compression interaction (MFCI) transformer, it consists of a modal feature compression module (MFC) and a modal feature interaction module (MFI).
• Figure 4: This is a visualization of the comparison experiment on BraTS2020 train dataset, From left to right is (a) T1, (b) T2, (c) T1ce, (d) Flair, (e) GT, (f) 3D U-Net, (g) Attention U-Net, (h) UNERT, (i) nnUnet, (j) TransBTS, (k) PANet, (l) CFCI-Net(Ours). To show the segmentation results more clearly, red represents necrotic tumor core (NCR), green represents peritumoral edema (ED), and blue code enhancing tumor (ET).
• Figure 5: This is a visualization of the ablation experiment on BraTS2020 train dataset, From left to right is (a)Flair, (b)GT, (c)baseline, (d)baseline_P, (e)MFCI, (f)SCFF, (g)CFCI-Net(Ours). To show the segmentation results more clearly, red represents necrotic tumor core (NCR), green represents peritumoral edema (ED), and blue code enhancing tumor (ET).