Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation

Saikat Roy, Gregor Koehler, Constantin Ulrich, Michael Baumgartner, Jens Petersen, Fabian Isensee, Paul F. Jaeger, Klaus Maier-Hein

TL;DR

MedNeXt introduces a fully ConvNeXt 3D segmentation framework tailored for medical data scarcity, combining residual inverted bottlenecks, UpKern large-kernel initialization, and compound scaling across depth, width, and receptive field. By adopting ConvNeXt-style blocks in an encoder–decoder architecture and enabling residual resampling, it preserves semantic richness across scales while training efficiently on limited datasets. Across four CT/MRI tasks with varying dataset sizes, MedNeXt achieves state-of-the-art or competitive results against strong baselines, with large-kernel variants further boosting performance; its results on public test sets demonstrate robust generalization without transfer learning. The work provides both empirical gains and practical tools (UpKern, detailed configurations, and code) to facilitate adoption and replication in medical image segmentation workflows.

Abstract

There has been exploding interest in embracing Transformer-based architectures for medical image segmentation. However, the lack of large-scale annotated medical datasets make achieving performances equivalent to those in natural images challenging. Convolutional networks, in contrast, have higher inductive biases and consequently, are easily trainable to high performance. Recently, the ConvNeXt architecture attempted to modernize the standard ConvNet by mirroring Transformer blocks. In this work, we improve upon this to design a modernized and scalable convolutional architecture customized to challenges of data-scarce medical settings. We introduce MedNeXt, a Transformer-inspired large kernel segmentation network which introduces - 1) A fully ConvNeXt 3D Encoder-Decoder Network for medical image segmentation, 2) Residual ConvNeXt up and downsampling blocks to preserve semantic richness across scales, 3) A novel technique to iteratively increase kernel sizes by upsampling small kernel networks, to prevent performance saturation on limited medical data, 4) Compound scaling at multiple levels (depth, width, kernel size) of MedNeXt. This leads to state-of-the-art performance on 4 tasks on CT and MRI modalities and varying dataset sizes, representing a modernized deep architecture for medical image segmentation. Our code is made publicly available at: https://github.com/MIC-DKFZ/MedNeXt.

MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation

TL;DR

MedNeXt introduces a fully ConvNeXt 3D segmentation framework tailored for medical data scarcity, combining residual inverted bottlenecks, UpKern large-kernel initialization, and compound scaling across depth, width, and receptive field. By adopting ConvNeXt-style blocks in an encoder–decoder architecture and enabling residual resampling, it preserves semantic richness across scales while training efficiently on limited datasets. Across four CT/MRI tasks with varying dataset sizes, MedNeXt achieves state-of-the-art or competitive results against strong baselines, with large-kernel variants further boosting performance; its results on public test sets demonstrate robust generalization without transfer learning. The work provides both empirical gains and practical tools (UpKern, detailed configurations, and code) to facilitate adoption and replication in medical image segmentation workflows.

Abstract

There has been exploding interest in embracing Transformer-based architectures for medical image segmentation. However, the lack of large-scale annotated medical datasets make achieving performances equivalent to those in natural images challenging. Convolutional networks, in contrast, have higher inductive biases and consequently, are easily trainable to high performance. Recently, the ConvNeXt architecture attempted to modernize the standard ConvNet by mirroring Transformer blocks. In this work, we improve upon this to design a modernized and scalable convolutional architecture customized to challenges of data-scarce medical settings. We introduce MedNeXt, a Transformer-inspired large kernel segmentation network which introduces - 1) A fully ConvNeXt 3D Encoder-Decoder Network for medical image segmentation, 2) Residual ConvNeXt up and downsampling blocks to preserve semantic richness across scales, 3) A novel technique to iteratively increase kernel sizes by upsampling small kernel networks, to prevent performance saturation on limited medical data, 4) Compound scaling at multiple levels (depth, width, kernel size) of MedNeXt. This leads to state-of-the-art performance on 4 tasks on CT and MRI modalities and varying dataset sizes, representing a modernized deep architecture for medical image segmentation. Our code is made publicly available at: https://github.com/MIC-DKFZ/MedNeXt.
Paper Structure (17 sections, 1 figure, 4 tables, 1 algorithm)

This paper contains 17 sections, 1 figure, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Fully ConvNeXt 3D Segmentation Architecture
  4. Resampling with Residual Inverted Bottlenecks
  5. UpKern: Large Kernel Convolutions without Saturation
  6. Compound Scaling of Depth, Width and Receptive Field
  7. Experimental Design
  8. Configurations, Implementation and Baselines
  9. Datasets
  10. Results and Discussion
  11. Performance ablation of architectural improvements
  12. Performance comparison to baselines
  13. Conclusion
  14. Supplementary Material
  15. UpKern: Algorithm
  16. ...and 2 more sections

Figures (1)

  • Figure 1: (a) Architectural design of the MedNeXt. The network has 4 Encoder and Decoder layers each, with a bottleneck layer. MedNeXt blocks are present in Up and Downsampling layers as well. Deep Supervision is used at each decoder layer, with lower loss weights at lower resolutions. All residuals are additive while convolutions are padded to retain tensor sizes. (b) Upsampled Kernel (UpKern) initialization of a pair of MedNeXt architectures with similar configurations ($\theta$) except kernel size ($k_1,k_2$). (c) MedNeXt-L ($5\times5\times5$) leaderboard performance.