D-Net: Dynamic Large Kernel with Dynamic Feature Fusion for Volumetric Medical Image Segmentation

TL;DR

D-Net is able to effectively utilize a multi-scale large receptive field and adaptively harness global contextual information and outperforms other state-of-the-art models in the two volumetric segmentation tasks, including abdominal multi-organ segmentation and multi-modality brain tumor segmentation.

Abstract

Hierarchical transformers have achieved significant success in medical image segmentation due to their large receptive field and capabilities of effectively leveraging global long-range contextual information. Convolutional neural networks (CNNs) can also deliver a large receptive field by using large kernels, enabling them to achieve competitive performance with fewer model parameters. However, CNNs incorporated with large convolutional kernels remain constrained in adaptively capturing multi-scale features from organs with large variations in shape and size due to the employment of fixed-sized kernels. Additionally, they are unable to utilize global contextual information efficiently. To address these limitations, we propose Dynamic Large Kernel (DLK) and Dynamic Feature Fusion (DFF) modules. The DLK module employs multiple large kernels with varying kernel sizes and dilation rates to capture multi-scale features. Subsequently, a dynamic selection mechanism is utilized to adaptively highlight the most important spatial features based on global information. Additionally, the DFF module is proposed to adaptively fuse multi-scale local feature maps based on their global information. We integrate DLK and DFF in a hierarchical transformer architecture to develop a novel architecture, termed D-Net. D-Net is able to effectively utilize a multi-scale large receptive field and adaptively harness global contextual information. Extensive experimental results demonstrate that D-Net outperforms other state-of-the-art models in the two volumetric segmentation tasks, including abdominal multi-organ segmentation and multi-modality brain tumor segmentation. Our code is available at https://github.com/sotiraslab/DLK.

Figures (8)

• Figure 1: The architecture of the D-Net, (a) DLK, (b) DFF, and (c) Channel Mixer. D-Net consists of an encoder, a bottleneck, a decoder, and a Salience layer. Two consecutive DLK blocks are used in each stage for feature extraction. Each DLK block consists of a DLK module and an MLP module. (a) After feature maps $\boldsymbol{X}_{in}^l$ are projected to $\boldsymbol{X}^l$, feature maps $\boldsymbol{X}_1^l$ and $\boldsymbol{X}_2^l$ are extracted by $5\times5\times5$ DWConv and $7\times7\times7$ DWConv, respectively. Subsequently, the dynamic selection values $w_1$ and $w_2$ are generated to calibrate features $\boldsymbol{X}_1^l$ and $\boldsymbol{X}_2^l$. These feature maps $\boldsymbol{X}_{ch}^l$ are scaled based on channel-wise importance. (b) The global channel information $w_{ch}$ is extracted from feature maps $\boldsymbol{F}_1^l$ and $\boldsymbol{F}_2^l$. These feature maps are calibrated to select informative features by a convolution layer as features $\boldsymbol{F}_{ch}^l$. The global spatial information $w_{sp}$ is extracted from $\boldsymbol{F}_1^l$ and $\boldsymbol{F}_2^l$, and is used to recalibrate features $\boldsymbol{F}_{ch}^l$ to generate the adaptively fused features $\hat{\boldsymbol{F}}^l$. (c) Input features $\boldsymbol{X}_{in}$ are mixed across channels to generate output features $\boldsymbol{X}_{out}$.
• Figure 2: The architecture of the DLK-Net. DLK-Net consists of an encoder, a bottleneck, and a decoder. Two consecutive DLK blocks are used in each stage for feature extraction. Each DLK block consists of a DLK module and an MLP module.
• Figure 3: The architecture of the DLK-NETR. DLK-NETR utilized the same encoder as DLK-Net and D-Net. In this encoder, two consecutive DLK blocks are used in each stage for feature extraction. Each DLK block consists of a DLK module and an MLP module. This encoder is incorporated into a hybrid CNN-ViT architecture which is the same as UNETR hatamizadeh2022unetr, Swin UNETR hatamizadeh2021swin, UX Net lee20223d, and VSmTrans liu2024vsmtrans.
• Figure 4: Qualitative comparison between D-Net and other methods ((c) VNet (d) nnU-Net (e) TransBTS (f) nnFormer (g) MedNext (f) SegFormer) across three public datasets, including the AMOS 2022 Multi-organ dataset, the MSD Brain Tumor dataset, and the MSD Hepatic Vessel Tumor dataset.
• Figure 5: Qualitative comparison between DLK-NETR and other methods which employ the same architecture ((c) UNETR (d) Swin UNETR (e) UX Net (f) VSmTrans) across three public datasets, including the AMOS 2022 Multi-organ dataset, the MSD Brain Tumor dataset, and the MSD Hepatic Vessel Tumor dataset.