Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Multi-Source Domain Adaptation through Gaussian Mixtures and Dataset Dictionary Learning

Eduardo Fernandes Montesuma, Stevan Le Stanc, Fred Ngolè Mboula

TL;DR

The paper tackles online multi-source domain adaptation where multiple heterogeneous sources must be aligned to a target domain that arrives as a data stream. It introduces an online Gaussian Mixture Modeling approach grounded in the Wasserstein geometry of Gaussian measures, and extends this with online DaDiL dictionary learning to memory-encode the target stream as a mixture of learned atoms. Key contributions include (i) an online GMM fitting and compression routine based on $W_2$ and Wasserstein barycenters, and (ii) a memory-enabled online dictionary learning framework that expresses target and sources as barycenters over a shared dictionary, enabling post-stream optimization. Empirical validation on the Tennessee Eastman Process benchmark demonstrates effective on-the-fly adaptation and memory-driven improvement after data streams end, suggesting practical impact for real-time fault diagnosis and similar online transfer tasks.

Abstract

This paper addresses the challenge of online multi-source domain adaptation (MSDA) in transfer learning, a scenario where one needs to adapt multiple, heterogeneous source domains towards a target domain that comes in a stream. We introduce a novel approach for the online fit of a Gaussian Mixture Model (GMM), based on the Wasserstein geometry of Gaussian measures. We build upon this method and recent developments in dataset dictionary learning for proposing a novel strategy in online MSDA. Experiments on the challenging Tennessee Eastman Process benchmark demonstrate that our approach is able to adapt \emph{on the fly} to the stream of target domain data. Furthermore, our online GMM serves as a memory, representing the whole stream of data.

Online Multi-Source Domain Adaptation through Gaussian Mixtures and Dataset Dictionary Learning

TL;DR

The paper tackles online multi-source domain adaptation where multiple heterogeneous sources must be aligned to a target domain that arrives as a data stream. It introduces an online Gaussian Mixture Modeling approach grounded in the Wasserstein geometry of Gaussian measures, and extends this with online DaDiL dictionary learning to memory-encode the target stream as a mixture of learned atoms. Key contributions include (i) an online GMM fitting and compression routine based on and Wasserstein barycenters, and (ii) a memory-enabled online dictionary learning framework that expresses target and sources as barycenters over a shared dictionary, enabling post-stream optimization. Empirical validation on the Tennessee Eastman Process benchmark demonstrates effective on-the-fly adaptation and memory-driven improvement after data streams end, suggesting practical impact for real-time fault diagnosis and similar online transfer tasks.

Abstract

This paper addresses the challenge of online multi-source domain adaptation (MSDA) in transfer learning, a scenario where one needs to adapt multiple, heterogeneous source domains towards a target domain that comes in a stream. We introduce a novel approach for the online fit of a Gaussian Mixture Model (GMM), based on the Wasserstein geometry of Gaussian measures. We build upon this method and recent developments in dataset dictionary learning for proposing a novel strategy in online MSDA. Experiments on the challenging Tennessee Eastman Process benchmark demonstrate that our approach is able to adapt \emph{on the fly} to the stream of target domain data. Furthermore, our online GMM serves as a memory, representing the whole stream of data.
Paper Structure (12 sections, 16 equations, 3 figures, 4 algorithms)

This paper contains 12 sections, 16 equations, 3 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Gaussian Mixture Models
  4. Optimal Transport
  5. Learning Theory and Domain Adaptation
  6. Proposed Method
  7. Online Gaussian Mixture Modeling
  8. Online GMM-DaDiL
  9. Experiments
  10. Online Gaussian Mixture Modeling
  11. Online Multi-Source Domain Adaptation
  12. Conclusion

Figures (3)

  • Figure 1: Illustration of the proposed compression mechanism. In (a), a novel batch of data arives, making $K > K_{max}$. As a result, we compute the pairwise Wasserstein distance between components (b). We then take the two closest components (a, in red), and merge them. In (c), we show in red the resulting component of the merging process.
  • Figure 2: Toy example illustrating the online learning of a under the Kullback-Leibler divergence acevedo2017multivariate (b, c), and the $\mathcal{W}_{2}$ distance (ours, e, f). Overall, using our strategy we achieve a fit an offline (d).
  • Figure 3: Reconstruction loss and classification accuracy of on mode 1 (target domain). Experiments are run independently on 5 folds of the target domain data. Solid lines represent the average, and the shaded regions represent $\pm 2\sigma$ around the mean.