Day-Night Adaptation: An Innovative Source-free Adaptation Framework for Medical Image Segmentation

TL;DR

DyNA addresses privacy-preserving medical image segmentation under distribution shifts by alternating day-time per-image low-frequency prompt adaptation with night-time self-training guided by a global student model. The day phase prevents error accumulation via a light-weight prompt and BN-statistics warm-up, while the night phase refines the model through self-training and EMA-based teacher-student updates, using test data stored during the day. Across OD/OC and polyp segmentation, DyNA consistently outperforms competing TTA and SFDA methods and remains robust across varying daytime data ratios. The approach offers practical impact for real-world deployment where data sharing is restricted and test data can be reused over time.

Abstract

Distribution shifts widely exist in medical images acquired from different medical centres, hindering the deployment of semantic segmentation models trained on one centre (source domain) to another (target domain). While unsupervised domain adaptation has shown significant promise in mitigating these shifts, it poses privacy risks due to sharing data between centres. To facilitate adaptation while preserving data privacy, source-free domain adaptation (SFDA) and test-time adaptation (TTA) have emerged as effective paradigms, relying solely on target domain data. However, SFDA requires a pre-collected target domain dataset before deployment. TTA insufficiently exploit the potential value of test data, as it processes the test data only once. Considering that most medical centres operate during the day and remain inactive at night in clinical practice, we propose a novel adaptation framework called Day-Night Adaptation (DyNA) with above insights, which performs adaptation through day-night cycles without requiring access to source data. During the day, a low-frequency prompt is trained to adapt the frozen model to each test sample. We construct a memory bank for prompt initialization and develop a warm-up mechanism to enhance prompt training. During the night, we reuse test data collected from the day and introduce a global student model to bridge the knowledge between teacher and student models, facilitating model fine-tuning while ensuring training stability. Extensive experiments demonstrate that our DyNA outperforms existing TTA and SFDA methods on two benchmark medical image segmentation tasks. Code will be available after the paper is published.

Figures (4)

• Figure 1: Pipeline of our DyNA framework. We continuously perform adaptation through day-night cycles. During the day, the pre-trained source model performs inference and adaptation on the test data, and the test data is stored in a data bank. During the night, the model's adaptation ability is further improved using the data bank.
• Figure 2: Overview of the adaptation approaches utilized during day and night in our DyNA framework: (1) During the day, we freeze the pre-trained source model and train a learnable prompt for each test sample to boost adaptation performance. The test data, trained prompts, and adapted predictions are collected for use in the adaptation at night. $\mu_s/\sigma_s$, $\mu_t/\sigma_t$, and $\mu_w/\sigma_w$ denote the source, test, and warm-up statistics (mean/standard deviation), respectively. (2) During the night, we freeze the trained prompts and fine-tune the source model using self-training on the collected data. A global student model is introduced to alleviate training instability. We integrate the pseudo-label and predictions of the global student model and teacher model to train the student model. The teacher model is updated by using the exponential moving average strategy and is subsequently frozen for inference the next day. 'BN': Abbreviation of 'Batch Normalization'. 'Aug.': Abbreviation of 'Augmentation'. 'GA': Abbreviation of 'Global Average'. 'EMA': Abbreviation of 'Exponential Moving Average'.
• Figure 3: Visualization of OD/OC segmentation results obtained by four competing TTA methods, four competing SFDA methods, and our DyNA. From top to bottom: segmentation results and ground truth obtained from the scenario denoted as "source $\rightarrow$ target". From left to right: segmentation results of the TTA methods (DLTTA, DUA, SAR, and DomainAdaptor), SFDA methods (FSM, UPL-SFDA, PETS, and PLPB), and our DyNA using different test data ratios, followed by the ground truth. To highlight potential over- or under-segmentation, the ground truth bounding boxes of OD and OC are overlaid on each segmentation result. Best viewed in color.
• Figure 4: Visualization of polyp segmentation results obtained by four competing TTA methods, four competing SFDA methods, and our DyNA. From top to bottom: segmentation results and ground truth obtained from the scenario denoted as "source $\rightarrow$ target". From left to right: segmentation results of the TTA methods (DLTTA, DUA, SAR, and DomainAdaptor), SFDA methods (FSM, UPL-SFDA, PETS, and PLPB), and our DyNA using different test data ratios, followed by the ground truth. To highlight potential over- or under-segmentation, the ground truth bounding boxes of foreground are overlaid on each segmentation result. Best viewed in color.