A Large Scale Benchmark for Test Time Adaptation Methods in Medical Image Segmentation

TL;DR

MedSeg-TTA tackles domain shift in medical image segmentation by introducing a large-scale, multi-modality benchmark that unifies data handling and test-time protocols. It systematically evaluates twenty representative TTA methods across seven imaging modalities and four adaptation paradigms, revealing that no single approach is universally best and that modality and task heavily influence performance. The study provides standardized datasets, reproducible implementations, and an open leaderboard to foster robust, clinically reliable TTA research, while highlighting challenges such as stability under large shifts and the need for principled parameter adaptation. Overall, the benchmark establishes a rigorous foundation for cross-modality evaluation and practical guidance for deploying test-time adaptive methods in clinical settings.

Abstract

Test time Adaptation is a promising approach for mitigating domain shift in medical image segmentation; however, current evaluations remain limited in terms of modality coverage, task diversity, and methodological consistency. We present MedSeg-TTA, a comprehensive benchmark that examines twenty representative adaptation methods across seven imaging modalities, including MRI, CT, ultrasound, pathology, dermoscopy, OCT, and chest X-ray, under fully unified data preprocessing, backbone configuration, and test time protocols. The benchmark encompasses four significant adaptation paradigms: Input-level Transformation, Feature-level Alignment, Output-level Regularization, and Prior Estimation, enabling the first systematic cross-modality comparison of their reliability and applicability. The results show that no single paradigm performs best in all conditions. Input-level methods are more stable under mild appearance shifts. Feature-level and Output-level methods offer greater advantages in boundary-related metrics, whereas prior-based methods exhibit strong modality dependence. Several methods degrade significantly under large inter-center and inter-device shifts, which highlights the importance of principled method selection for clinical deployment. MedSeg-TTA provides standardized datasets, validated implementations, and a public leaderboard, establishing a rigorous foundation for future research on robust, clinically reliable test-time adaptation. All source codes and open-source datasets are available at https://github.com/wenjing-gg/MedSeg-TTA.

Figures (18)

• Figure 1: Schematic diagram of the four paradigms in TTA. (a) Input-level Transformation, adapting the model to target domain images by altering image appearances; (b) Feature-level Alignment, aligning features from the source and target domains; (c) Output-level Regularization, regularizing the model outputs for target domain adaptation; (d) Prior Estimation, forcing the model to update towards the target domain using prior information of the target domain.
• Figure 2: The overall workflow of TTA methods for medical image segmentation. It begins with the collection of multi-center data across seven modalities, which include MRI, CT, US, PATH, DER, OCT, and CXR. Next, a pretrained network is trained on source domain data, and benchmark testing is conducted on the target domain. Finally, out-of-domain segmentation performance improves with TTA methods.
• Figure 3: Overview of the multi-institutional benchmark and dataset construction pipeline. (a) participating institutions contributing data for the benchmark;(b) unified construction pipeline comprising dataset integration, integrity checking, data cleaning, data processing, and intensity standardization; and (c) data preprocessing procedures, including CT volume stacking from DICOM slices to 3D volumes, PATH patch cropping from whole-slide images, and source–target label filtering to harmonize label sets. (d) distribution of datasets across imaging modalities and their assignment to source or target domains;
• Figure 4: Multi-sequence MRI for brain tumor segmentation. Cases from the source dataset BraTS-GLI2024 and the target dataset BraTS-SSA. For the source dataset area, the first row displays T1, T1-weighted contrast-enhanced, T2 and FLAIR sequence images and enhancing tumor, tumor core and whole tumor annotations from anatomical views including axial, coronal and sagittal planes; the second row presents the corresponding annotation masks. The bottom section shows the equivalent multi-sequence MRI images and annotations for the target dataset.
• Figure 5: CT volumes for liver segmentation. Cases from the source dataset LiTS and the target dataset 3D-IRCADB. For the source dataset area, the first row displays CT images and liver-tumor annotations from anatomical views, including axial and coronal planes; the second row presents the corresponding annotation masks. The bottom section shows the equivalent CT images and annotations for the target dataset.