Med-Tuning: A New Parameter-Efficient Tuning Framework for Medical Volumetric Segmentation

TL;DR

This paper tackles the high cost of full fine-tuning for medical volumetric segmentation by introducing Med-Tuning, a parameter-efficient tuning framework that freezes a 2D Transformer backbone pre-trained on natural images and inserts Med-Adapters for task-specific feature extraction. The Med-Adapter combines intra-stage multi-scale local features and a fast FFT-based global branch with an inter-stage interaction mechanism to leverage volumetric intra- and inter-slice correlations while keeping tunable parameters to a minimum. Empirical results on KiTS 2019 and BraTS 2019/2020 show Med-Tuning achieves better Dice scores than full fine-tuning and prior PET methods with up to roughly a 4x reduction in tuned parameters, demonstrating strong efficiency and accuracy gains. The approach generalizes across different pre-trained weights, 3D baselines, and even medical-image pre-training, indicating practical utility for deploying robust medical segmentation models in resource-constrained settings. Overall, Med-Tuning offers a scalable path to adapting large visual foundation models to medical volumes with high performance and low training cost.

Abstract

The "pre-training then fine-tuning (FT)" paradigm is widely adopted to boost the model performance of deep learning-based methods for medical volumetric segmentation. However, conventional full FT incurs high computational and memory costs. Thus, it is of increasing importance to fine-tune pre-trained models for medical volumetric segmentation tasks in a both effective and parameter-efficient manner. In this paper, we introduce a new framework named Med-Tuning to realize parameter-efficient tuning (PET) for medical volumetric segmentation task and an efficient plug-and-play module named Med-Adapter for task-specific feature extraction. With a small number of tuned parameters, our framework enhances the 2D baselines's precision on segmentation tasks, which are pre-trained on natural images. Extensive experiments on three benchmark datasets (CT and MRI modalities) show that our method achieves better results than previous PET methods on volumetric segmentation tasks. Compared to full FT, Med-Tuning reduces the fine-tuned model parameters by up to 4x, with even better segmentation performance. Our project webpage is at \url{https://rubics-xuan.github.io/Med-Tuning/}.

Figures (5)

• Figure 1: The two-fold gaps between upstream pre-training and our downstream fine-tuning.
• Figure 2: The overall architecture of Med-Tuning. Med-Tuning consists of a 2D Transformer baseline with proposed Med-Adapters inserted at each encoder stage. Only Med-Adapters and decoder are tuned while all the other layers stay frozen.
• Figure 3: Comparison with previous PET methods in terms of the number of tuned parameters and Dice scores.
• Figure 4: The visual comparison of segmentation results on BraTS 2019. The blue, red and green regions denote the enhancing tumors, non-enhancing tumors, and peritumoral edema. GT=Ground Truth.
• Figure 5: The visual comparison of segmentation results on KiTS 2019. The red and green regions denote the kidneys and kidney tumors. GT=Ground Truth.