AFFSegNet: Adaptive Feature Fusion Segmentation Network for Microtumors and Multi-Organ Segmentation

TL;DR

AFFSegNet tackles the challenge of accurately segmenting microtumors and small organs by fusing local and global context through a hybrid transformer architecture. It combines a residual U-shaped encoder with a Multi-scale Window Attention (MWA) block and an Adaptive Feature Fusion (AFF) decoder consisting of Long Range Dependencies (LRD), Multi-Scale Feature Fusion (MFF), and Adaptive Semantic Center (ASC) blocks. The model achieves state-of-the-art results on LiTS2017, ISICDM2019, and Synapse, demonstrating strong cross-domain generalization and precise boundary delineation for small structures. This approach offers a scalable, effective solution for clinically impactful medical image segmentation with potential to aid diagnosis and treatment planning.

Abstract

Medical image segmentation, a crucial task in computer vision, facilitates the automated delineation of anatomical structures and pathologies, supporting clinicians in diagnosis, treatment planning, and disease monitoring. Notably, transformers employing shifted window-based self-attention have demonstrated exceptional performance. However, their reliance on local window attention limits the fusion of local and global contextual information, crucial for segmenting microtumors and miniature organs. To address this limitation, we propose the Adaptive Semantic Segmentation Network (ASSNet), a transformer architecture that effectively integrates local and global features for precise medical image segmentation. ASSNet comprises a transformer-based U-shaped encoder-decoder network. The encoder utilizes shifted window self-attention across five resolutions to extract multi-scale features, which are then propagated to the decoder through skip connections. We introduce an augmented multi-layer perceptron within the encoder to explicitly model long-range dependencies during feature extraction. Recognizing the constraints of conventional symmetrical encoder-decoder designs, we propose an Adaptive Feature Fusion (AFF) decoder to complement our encoder. This decoder incorporates three key components: the Long Range Dependencies (LRD) block, the Multi-Scale Feature Fusion (MFF) block, and the Adaptive Semantic Center (ASC) block. These components synergistically facilitate the effective fusion of multi-scale features extracted by the decoder while capturing long-range dependencies and refining object boundaries. Comprehensive experiments on diverse medical image segmentation tasks, including multi-organ, liver tumor, and bladder tumor segmentation, demonstrate that ASSNet achieves state-of-the-art results. Code and models are available at: \url{https://github.com/lzeeorno/ASSNet}.

Figures (4)

• Figure 1: Overview of the AFFSegNet architecture.
• Figure 2: This figure presents details of a schematic diagram of the proposed Multi-scale Window Attention (MWA) transformer block.
• Figure 3: This figure presents details of a schematic diagram of the proposed Adaptive Feature Fusion (AFF) Decoder.
• Figure 4: LiTS2017, ISICDM2019 and Synapse Prediction Results