Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment

Théo Gnassounou, Antoine Collas, Rémi Flamary, Karim Lounici, Alexandre Gramfort

TL;DR

Numerical experiments on multivariate biosignals and image data show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings, and demonstrates the efficiency of the proposed computational schema.

Abstract

Machine learning applications on signals such as computer vision or biomedical data often face significant challenges due to the variability that exists across hardware devices or session recordings. This variability poses a Domain Adaptation (DA) problem, as training and testing data distributions often differ. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. This Optimal Transport (OT) based method adapts the cross-power spectrum density (cross-PSD) of multivariate signals by mapping them to the Wasserstein barycenter of source domains (multi-source DA). Predictions for new domains can be done with a filtering without the need for retraining a model with source data (test-time DA). We also study and discuss two special cases of the method, Temporal Monge Alignment (TMA) and Spatial Monge Alignment (SMA). Non-asymptotic concentration bounds are derived for the mappings estimation, which reveals a bias-plus-variance error structure with a variance decay rate of $\mathcal{O}(n_\ell^{-1/2})$ with $n_\ell$ the signal length. This theoretical guarantee demonstrates the efficiency of the proposed computational schema. Numerical experiments on multivariate biosignals and image data show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings. Notably, STMA is a pre-processing step complementary to state-of-the-art deep learning methods.

Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment

TL;DR

Numerical experiments on multivariate biosignals and image data show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings, and demonstrates the efficiency of the proposed computational schema.

Abstract

Machine learning applications on signals such as computer vision or biomedical data often face significant challenges due to the variability that exists across hardware devices or session recordings. This variability poses a Domain Adaptation (DA) problem, as training and testing data distributions often differ. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. This Optimal Transport (OT) based method adapts the cross-power spectrum density (cross-PSD) of multivariate signals by mapping them to the Wasserstein barycenter of source domains (multi-source DA). Predictions for new domains can be done with a filtering without the need for retraining a model with source data (test-time DA). We also study and discuss two special cases of the method, Temporal Monge Alignment (TMA) and Spatial Monge Alignment (SMA). Non-asymptotic concentration bounds are derived for the mappings estimation, which reveals a bias-plus-variance error structure with a variance decay rate of with the signal length. This theoretical guarantee demonstrates the efficiency of the proposed computational schema. Numerical experiments on multivariate biosignals and image data show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings. Notably, STMA is a pre-processing step complementary to state-of-the-art deep learning methods.
Paper Structure (78 sections, 11 theorems, 110 equations, 13 figures, 5 tables, 2 algorithms)

This paper contains 78 sections, 11 theorems, 110 equations, 13 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Domain Adaptation (DA) from single to multi-domain
  3. DA for multivariate signals and biomedical data
  4. Contributions
  5. Paper outline
  6. Signal adaptation with Optimal Transport
  7. Optimal Transport between Gaussian distributions
  8. Monge mapping for Gaussian distributions
  9. Wasserstein barycenter between Gaussian distributions
  10. OT mapping of stationary signals using circulant matrices
  11. Circulant matrices for stationary and periodic signals
  12. Convolutional Monge Mapping and filter size
  13. Approximation of circulant matrices
  14. Spatio-Temporal Monge mapping and barycenter for Gaussian signals
  15. Spatio-temporal covariance matrix
  16. ...and 63 more sections

Key Result

Proposition 1

Let $\mathbf{F}_{n_\ell} \in \mathbb{C}^{n_\ell\times n_\ell}$ and $\mathbf{F}_{f} \in \mathbb{C}^{f\times f}$ be the Fourier bases. Given $\mathbf{A} = \mathbf{F}_{n_\ell} \mathop{\mathrm{diag}}\nolimits(\mathbf{q}) \mathbf{F}_{n_\ell}^{\mathsf{H}} \in \mathcal{E}_{f,n_\ell}$ with $\mathbf{q} \in \ where $*$ is the convolution operator and $\mathbf{h}$ is the inverse Fourier transform of $\mathbf

Figures (13)

  • Figure 1: Illustration of Monge Alignment. At train-time the barycenter (grey ellipse) is estimated from 3 source distributions (colored ellipses). The predictor is learned on normalized data. At test-time the same barycenter is used to align the unlabeled target distribution (orange ellipse) and predict.
  • Figure 2: A Monge mapping $\mathbf{A}$ (cf. \ref{['eq:Monge_map']}) that is a $5 \times 5$ symmetric positive definite and circulant matrix, i.e., belonging to $\mathcal{C}_5 \cap \mathcal{S}_5^{++}$, and its approximation in $\mathcal{E}_{3,5} \cap \mathcal{S}_5^{++}$ as defined in \ref{['eq:def_approx']}. In this case, for $\mathbf{x} \in \mathbb{R}^{n_\ell}$, $\mathcal{P}_{f}(\mathbf{A}) \mathbf{x} = \mathbf{h} * \mathbf{x}$ with $\mathbf{h} = [ \color{Orange}\mdblksquare\color{Blue} \mdblksquare\color{Orange}\mdblksquare\color{black}]^{\mkern-1.5mu\mathsf{T}}$.
  • Figure 3: Block diagonalization of the covariance matrix ${\boldsymbol\Sigma} \in \mathcal{S}_{n_c n_\ell}^{++}$ following the Assumption \ref{['assu:spat_temp']} with $n_\ell=5$ and $n_c = 3$. $\mathbf{F} = \mathop{\mathrm{diag}}\nolimits(\mathbf{F}_{n_\ell}, \dots, \mathbf{F}_{n_\ell}) \in \mathbb{C}^{n_c n_\ell}$ and $\mathbf{U}$ is a permutation matrix.
  • Figure 4: Illustration of the Spatio-Temporal Monge mapping on sleep data. (a) Cross-PSD alignment from source cross-PSD (green dotted line) to target cross-PSD (blue dotted line) for different filter sizes. (b) Bures-Wasserstein distance between the aligned signal and the source signal (green line) and the target line (blue line).
  • Figure 5: Illustration of the STMA method on sleep staging data. (a) The cross-PSD of the barycenter ($\boldsymbol{--}$) is computed from all the cross-PSD of the source domains. (b) The cross-PSDs of the source are aligned with the Monge mapping with a small filter size. (c) The cross-PSDs of the source are aligned with the Monge mapping with a big filter size. The bigger the filter size is, the more the cross-PSD are well aligned.
  • ...and 8 more figures

Theorems & Definitions (12)

  • Proposition 1: $\mathcal{E}_{f, n_\ell}$ and filtering
  • Definition 3
  • Proposition 4: Spatio-Temporal mapping
  • Lemma 5: Spatio-Temporal barycenter
  • Proposition 7: Temporal mapping
  • Lemma 8: Temporal barycenter
  • Proposition 10: Spatial mapping
  • Lemma 11: Spatial barycenter
  • Theorem 12: STMA concentration bound
  • Theorem 13: TMA concentration bound
  • ...and 2 more