Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-source-free Domain Adaptation via Uncertainty-aware Adaptive Distillation

Yaxuan Song, Jianan Fan, Dongnan Liu, Weidong Cai

TL;DR

This work tackles multi-source-free unsupervised domain adaptation for medical imaging by introducing Uncertainty-aware Adaptive Distillation (UAD), a two-level framework that selects a source model most aligned with the target data (model-level) and assigns pseudo-labels from the most confident source per instance (instance-level). It incorporates Temperature Scaling to calibrate source-model confidences, using calibrated logits $z_j = \theta_S^{j}(x_T)/\mathcal{T}_{j}$ and a margin-based confidence measure $\mathcal{M} = \mathrm{Top1}(\delta(\theta(x))) - \mathrm{Top2}(\delta(\theta(x)))$ to guide both initialization and pseudo-labeling. The method is validated on two medical benchmarks—the multi-centre Diabetic Retinopathy and HAM10000 skin lesion datasets—demonstrating significant gains over existing SFDA/MSFDA approaches, with ablations showing the complementary benefits of model-level and instance-level UAD and the positive impact of TS calibration. The accompanying codebase supports reproducibility and practical deployment in privacy-preserving, multi-centre clinical settings.

Abstract

Source-free domain adaptation (SFDA) alleviates the domain discrepancy among data obtained from domains without accessing the data for the awareness of data privacy. However, existing conventional SFDA methods face inherent limitations in medical contexts, where medical data are typically collected from multiple institutions using various equipment. To address this problem, we propose a simple yet effective method, named Uncertainty-aware Adaptive Distillation (UAD) for the multi-source-free unsupervised domain adaptation (MSFDA) setting. UAD aims to perform well-calibrated knowledge distillation from (i) model level to deliver coordinated and reliable base model initialisation and (ii) instance level via model adaptation guided by high-quality pseudo-labels, thereby obtaining a high-performance target domain model. To verify its general applicability, we evaluate UAD on two image-based diagnosis benchmarks among two multi-centre datasets, where our method shows a significant performance gain compared with existing works. The code is available at https://github.com/YXSong000/UAD.

Multi-source-free Domain Adaptation via Uncertainty-aware Adaptive Distillation

TL;DR

This work tackles multi-source-free unsupervised domain adaptation for medical imaging by introducing Uncertainty-aware Adaptive Distillation (UAD), a two-level framework that selects a source model most aligned with the target data (model-level) and assigns pseudo-labels from the most confident source per instance (instance-level). It incorporates Temperature Scaling to calibrate source-model confidences, using calibrated logits and a margin-based confidence measure to guide both initialization and pseudo-labeling. The method is validated on two medical benchmarks—the multi-centre Diabetic Retinopathy and HAM10000 skin lesion datasets—demonstrating significant gains over existing SFDA/MSFDA approaches, with ablations showing the complementary benefits of model-level and instance-level UAD and the positive impact of TS calibration. The accompanying codebase supports reproducibility and practical deployment in privacy-preserving, multi-centre clinical settings.

Abstract

Source-free domain adaptation (SFDA) alleviates the domain discrepancy among data obtained from domains without accessing the data for the awareness of data privacy. However, existing conventional SFDA methods face inherent limitations in medical contexts, where medical data are typically collected from multiple institutions using various equipment. To address this problem, we propose a simple yet effective method, named Uncertainty-aware Adaptive Distillation (UAD) for the multi-source-free unsupervised domain adaptation (MSFDA) setting. UAD aims to perform well-calibrated knowledge distillation from (i) model level to deliver coordinated and reliable base model initialisation and (ii) instance level via model adaptation guided by high-quality pseudo-labels, thereby obtaining a high-performance target domain model. To verify its general applicability, we evaluate UAD on two image-based diagnosis benchmarks among two multi-centre datasets, where our method shows a significant performance gain compared with existing works. The code is available at https://github.com/YXSong000/UAD.
Paper Structure (11 sections, 6 equations, 1 figure, 1 table)

This paper contains 11 sections, 6 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Problem Setting
  4. Uncertainty-aware Adaptive Distillation
  5. Temperature Scaling
  6. Experiments and Results
  7. Dataset and Implementation Details
  8. Comparison Experiments
  9. Ablation Study
  10. Conclusion
  11. Compliance with Ethical Standards

Figures (1)

  • Figure 1: Overview of the proposed framework. Our framework follows a multi-source domain model pre-training process with a two-stage uncertainty-aware adaptive distillation (UAD) process of model initialisation and pseudo-labelling.