MetaFormer-driven Encoding Network for Robust Medical Semantic Segmentation

TL;DR

The paper addresses the need for accurate medical image segmentation under limited compute. It introduces MFEnNet, a MetaFormer-driven encoding network that replaces self-attention with pooling token mixers inside a U-Net backbone, complemented by Swish activation and a spatial pyramid pooling bottleneck to capture multi-scale context efficiently. Key contributions include adapting MetaFormer to medical segmentation, using pooling-based token mixing to reduce cost while maintaining performance, and demonstrating competitive accuracy with substantially lower parameters and FLOPs on CASIA-Iris and ISBI datasets. The work demonstrates a practical pathway toward robust, edge-friendly medical segmentation suitable for resource-constrained clinical environments.

Abstract

Semantic segmentation is crucial for medical image analysis, enabling precise disease diagnosis and treatment planning. However, many advanced models employ complex architectures, limiting their use in resource-constrained clinical settings. This paper proposes MFEnNet, an efficient medical image segmentation framework that incorporates MetaFormer in the encoding phase of the U-Net backbone. MetaFormer, an architectural abstraction of vision transformers, provides a versatile alternative to convolutional neural networks by transforming tokenized image patches into sequences for global context modeling. To mitigate the substantial computational cost associated with self-attention, the proposed framework replaces conventional transformer modules with pooling transformer blocks, thereby achieving effective global feature aggregation at reduced complexity. In addition, Swish activation is used to achieve smoother gradients and faster convergence, while spatial pyramid pooling is incorporated at the bottleneck to improve multi-scale feature extraction. Comprehensive experiments on different medical segmentation benchmarks demonstrate that the proposed MFEnNet approach attains competitive accuracy while significantly lowering computational cost compared to state-of-the-art models. The source code for this work is available at https://github.com/tranleanh/mfennet.

Figures (5)

• Figure 1: The trade-off between accuracy (IoU) vs complexity (FLOPs).
• Figure 2: The proposed network.
• Figure 3: Examples of data used in the experiment.
• Figure 4: Typical visual comparisons of various approaches on CASIA data.
• Figure 5: Typical visual comparisons of various approaches on ISBI data.