Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Continuous Domain Adaptation with Multi-Path Transfer Curriculum

Hanbing Liu, Jingge Wang, Xuan Zhang, Ye Guo, Yang Li

TL;DR

This work tackles large distribution shifts in CDA by introducing W-MPOT, which combines a Wasserstein-based transfer curriculum for ordering intermediate domains with a Multi-Path Optimal Transport framework to transfer knowledge from the source to the target along multiple paths. The curriculum provides a principled, metadata-free ordering via $W_k$, while the MPOT module enforces bidirectional path consistency to mitigate error accumulation during sequential transfers. Empirical results on ADNI, Battery, and Rotated MNIST demonstrate significant gains over DOT and traditional COT, including up to $54.1\%$ accuracy improvement in Alzheimer’s MRI classification and $94.7\%$ MSE reduction in battery capacity estimation. The approach advances CDA by addressing both domain ordering without metadata and robustness to cumulative transfer errors, with potential impact in healthcare and energy systems.

Abstract

Addressing the large distribution gap between training and testing data has long been a challenge in machine learning, giving rise to fields such as transfer learning and domain adaptation. Recently, Continuous Domain Adaptation (CDA) has emerged as an effective technique, closing this gap by utilizing a series of intermediate domains. This paper contributes a novel CDA method, W-MPOT, which rigorously addresses the domain ordering and error accumulation problems overlooked by previous studies. Specifically, we construct a transfer curriculum over the source and intermediate domains based on Wasserstein distance, motivated by theoretical analysis of CDA. Then we transfer the source model to the target domain through multiple valid paths in the curriculum using a modified version of continuous optimal transport. A bidirectional path consistency constraint is introduced to mitigate the impact of accumulated mapping errors during continuous transfer. We extensively evaluate W-MPOT on multiple datasets, achieving up to 54.1\% accuracy improvement on multi-session Alzheimer MR image classification and 94.7\% MSE reduction on battery capacity estimation.

Enhancing Continuous Domain Adaptation with Multi-Path Transfer Curriculum

TL;DR

This work tackles large distribution shifts in CDA by introducing W-MPOT, which combines a Wasserstein-based transfer curriculum for ordering intermediate domains with a Multi-Path Optimal Transport framework to transfer knowledge from the source to the target along multiple paths. The curriculum provides a principled, metadata-free ordering via , while the MPOT module enforces bidirectional path consistency to mitigate error accumulation during sequential transfers. Empirical results on ADNI, Battery, and Rotated MNIST demonstrate significant gains over DOT and traditional COT, including up to accuracy improvement in Alzheimer’s MRI classification and MSE reduction in battery capacity estimation. The approach advances CDA by addressing both domain ordering without metadata and robustness to cumulative transfer errors, with potential impact in healthcare and energy systems.

Abstract

Addressing the large distribution gap between training and testing data has long been a challenge in machine learning, giving rise to fields such as transfer learning and domain adaptation. Recently, Continuous Domain Adaptation (CDA) has emerged as an effective technique, closing this gap by utilizing a series of intermediate domains. This paper contributes a novel CDA method, W-MPOT, which rigorously addresses the domain ordering and error accumulation problems overlooked by previous studies. Specifically, we construct a transfer curriculum over the source and intermediate domains based on Wasserstein distance, motivated by theoretical analysis of CDA. Then we transfer the source model to the target domain through multiple valid paths in the curriculum using a modified version of continuous optimal transport. A bidirectional path consistency constraint is introduced to mitigate the impact of accumulated mapping errors during continuous transfer. We extensively evaluate W-MPOT on multiple datasets, achieving up to 54.1\% accuracy improvement on multi-session Alzheimer MR image classification and 94.7\% MSE reduction on battery capacity estimation.
Paper Structure (17 sections, 1 theorem, 14 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 17 sections, 1 theorem, 14 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Preliminary
  4. Method Framework
  5. Wasserstein-based Transfer Curriculum
  6. Multi-Path Optimal Transport
  7. Experimental Results
  8. Datasets and Experimental Configurations
  9. Analysis of Wasserstein-based Transfer Curriculum
  10. Adaptation Comparison Results
  11. Ablation Study
  12. Conclusion
  13. Acknowledgement
  14. Optimal Transport
  15. Lemma of Generalization Bound
  16. ...and 2 more sections

Key Result

lemma thmcounterlemma

Suppose the two domains have the same true labeling function $f:\mathcal{X}\rightarrow [0,1]$. Let $\mu_S$, $\mu_T\in \mathcal{P}(\mathcal{X})$ be two probability measures on $\mathbb{R}^d$. Assume that all hypotheses in the hypothesis set $H$ satisfies the $A$-Lipschitz continuous condition for som where $\epsilon$ is the combined error of the ideal hypothesis $h^*$ that minimizes the combined er

Figures (4)

  • Figure 1: Illustration of our proposed W-MPOT. (a) In the Rotated MNIST example, we are given a source domain of 0 degrees and a target/intermediate domain set with unknown degrees. (b) In our proposed W-MPOT, we address the challenge of unknown domain metadata (angles) and perform CDA. We utilize Wasserstein based transfer curriculum to sort intermediate domains and employ MPOT to enforce consistency, thereby enhancing the transfer effectiveness.
  • Figure 2: The Effect of Multi-Path regularization on the Optimal Transport of source domain sample. Visual experiments are conducted on simulated half-moon data to compare the migration effects of (a) COT with our proposed (b) MPOT. The source domain has an angle of 0 degrees, and the angles on the graph increase by 36 degrees from left to right. (a) and (b) depict the mapping results obtained by applying the COT and MPOT methods respectively to sequentially map the source domain to the target domains. The triangles represent the ground truth target domain data, while the plus signs represent the mapped source domain data after the adaptation.
  • Figure 3: Domain ordering Results. The results for the (a)Rotated MNIST, (b)ADNI, and (c)Battery Charging-discharging Capacity datasets are shown in this figure. The evaluation metric for Rotated MNIST and ADNI is accuracy, while for the Battery dataset is MSE. The methods compared are DOT (Direct Optimal Transport) and COT (Continuous Optimal Transport). Two sorting approaches are utilized: metadata-based sorting and Wasserstein based transfer curriculum. For ease of comparison, the y-coordinate of the first two datasets represents $1-ACC$, where lower values indicate superior performance. Each configuration is evaluated 100 times, and the shaded areas represent the variance.
  • Figure 4: Ablation study Results. The experiments from a to b involved domain partitioning, sorting strategy, and path consistency. For each number of intermediate domains in a and b, the experiment was randomly repeated 100 times. The solid line represents the mean MSE, while the shaded area represents the variance. The experiment in c is conducted by fixing the number of intermediate domains to 2 and randomly sampling different intermediate domains 100 times.

Theorems & Definitions (1)

  • lemma thmcounterlemma