Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Domain Adaptation for Time-Series Classification to Mitigate Covariate Shift

Felix Ott, David Rügamer, Lucas Heublein, Bernd Bischl, Christopher Mutschler

TL;DR

This work tackles covariate shift in time-series classification by proposing a two-step supervised domain adaptation framework. It first learns class-specific transformations from a small source domain to a large target domain using optimal transport (EMD/Sinkhorn) and correlation alignment (CORAL), then selects the best transformation at inference via embedding similarity metrics. Across synthetic sinusoidal data and online handwriting (OnHW) recognition datasets, the approach yields substantial improvements over transfer learning, with Sinkhorn-based OT combined with a kernelized MMD distance offering the best accuracy-runtimes trade-off. The method enables fast, per-class adaptation to new writers or domains without full re-training, making it practical for embedded deployment and real-time DA in time-series tasks.

Abstract

The performance of a machine learning model degrades when it is applied to data from a similar but different domain than the data it has initially been trained on. To mitigate this domain shift problem, domain adaptation (DA) techniques search for an optimal transformation that converts the (current) input data from a source domain to a target domain to learn a domain-invariant representation that reduces domain discrepancy. This paper proposes a novel supervised DA based on two steps. First, we search for an optimal class-dependent transformation from the source to the target domain from a few samples. We consider optimal transport methods such as the earth mover's distance, Sinkhorn transport and correlation alignment. Second, we use embedding similarity techniques to select the corresponding transformation at inference. We use correlation metrics and higher-order moment matching techniques. We conduct an extensive evaluation on time-series datasets with domain shift including simulated and various online handwriting datasets to demonstrate the performance.

Domain Adaptation for Time-Series Classification to Mitigate Covariate Shift

TL;DR

This work tackles covariate shift in time-series classification by proposing a two-step supervised domain adaptation framework. It first learns class-specific transformations from a small source domain to a large target domain using optimal transport (EMD/Sinkhorn) and correlation alignment (CORAL), then selects the best transformation at inference via embedding similarity metrics. Across synthetic sinusoidal data and online handwriting (OnHW) recognition datasets, the approach yields substantial improvements over transfer learning, with Sinkhorn-based OT combined with a kernelized MMD distance offering the best accuracy-runtimes trade-off. The method enables fast, per-class adaptation to new writers or domains without full re-training, making it practical for embedded deployment and real-time DA in time-series tasks.

Abstract

The performance of a machine learning model degrades when it is applied to data from a similar but different domain than the data it has initially been trained on. To mitigate this domain shift problem, domain adaptation (DA) techniques search for an optimal transformation that converts the (current) input data from a source domain to a target domain to learn a domain-invariant representation that reduces domain discrepancy. This paper proposes a novel supervised DA based on two steps. First, we search for an optimal class-dependent transformation from the source to the target domain from a few samples. We consider optimal transport methods such as the earth mover's distance, Sinkhorn transport and correlation alignment. Second, we use embedding similarity techniques to select the corresponding transformation at inference. We use correlation metrics and higher-order moment matching techniques. We conduct an extensive evaluation on time-series datasets with domain shift including simulated and various online handwriting datasets to demonstrate the performance.
Paper Structure (37 sections, 1 theorem, 10 equations, 71 figures, 9 tables, 1 algorithm)

This paper contains 37 sections, 1 theorem, 10 equations, 71 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Domain Adaptation for Time-Series Classification.
  4. Covariance- and Discrepancy-based Methods for DA
  5. Optimal Transport for DA
  6. Embedding Distances.
  7. Methodology
  8. Notation
  9. MTS Classification.
  10. Domain Adaptation.
  11. Method Overview
  12. Embedding Transformation
  13. Homogeneous Domain Adaptation.
  14. Optimal Transport.
  15. Transformation Selection for Inference
  16. ...and 22 more sections

Key Result

Theorem 1

Let $\mathcal{H}$ be a hypothesis space. Given the target domain $\mathcal{D}_T$ and source domain $\mathcal{D}_S$, we have where $\mathcal{R}_T(h)$ is the target domain error and $\mathcal{R}_S(h)$ is the source domain error. $d_{\mathcal{H}\Delta\mathcal{H}}$ is the discrepancy distance between $\mathcal{D}_S$ and $\mathcal{D}_T$ w.r.t. $\mathcal{H}$. Then, given the label functions $g_T$ and $

Figures (71)

  • Figure 1: Domain adaptation. To compensate the domain shift the source data is transformed into the target data (1st: 2D source and target domain features, before transformation, 2nd: after transformation). 3rd and 4th: their distributions.
  • Figure 2: Domain-variant representation of MTS data of a source and target sample from a sensor pen representing the label '5' in an online handwriting task. Note that both samples have different number of time steps $t_S$ (left) and $t_T$ (right). Top row: data from an accelerometer; middle row: gyroscope measures; bottom row: data from a force sensor.
  • Figure 3: Method overview. We transform the feature embeddings of the source domain (training data, few samples) into feature embeddings of the target domain (training adaptation data, many samples) with optimal transport and correlation alignment. For inference, we evaluate the feature similarity.
  • Figure 4: Target domain (training) data $\mathcal{U}_{T_t}$ with noise $b = 0.4$.
  • Figure 5: Source domain (training) data $\mathcal{U}_{S_t}$ with noise $b = 0.2$.
  • ...and 66 more figures

Theorems & Definitions (1)

  • Theorem 1: Zhang, 2021 zhang