Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

POND: Multi-Source Time Series Domain Adaptation with Information-Aware Prompt Tuning

Junxiang Wang, Guangji Bai, Wei Cheng, Zhengzhang Chen, Liang Zhao, Haifeng Chen

TL;DR

PrOmpt-based domaiN Discrimination (POND) is introduced, the first framework to utilize prompts for time series domain adaptation and two criteria to select good prompts are proposed, which are used to choose the most suitable source domain for domain adaptation.

Abstract

Time series domain adaptation stands as a pivotal and intricate challenge with diverse applications, including but not limited to human activity recognition, sleep stage classification, and machine fault diagnosis. Despite the numerous domain adaptation techniques proposed to tackle this complex problem, they primarily focus on domain adaptation from a single source domain. Yet, it is more crucial to investigate domain adaptation from multiple domains due to the potential for greater improvements. To address this, three important challenges need to be overcome: 1). The lack of exploration to utilize domain-specific information for domain adaptation, 2). The difficulty to learn domain-specific information that changes over time, and 3). The difficulty to evaluate learned domain-specific information. In order to tackle these challenges simultaneously, in this paper, we introduce PrOmpt-based domaiN Discrimination (POND), the first framework to utilize prompts for time series domain adaptation. Specifically, to address Challenge 1, we extend the idea of prompt tuning to time series analysis and learn prompts to capture common and domain-specific information from all source domains. To handle Challenge 2, we introduce a conditional module for each source domain to generate prompts from time series input data. For Challenge 3, we propose two criteria to select good prompts, which are used to choose the most suitable source domain for domain adaptation. The efficacy and robustness of our proposed POND model are extensively validated through experiments across 50 scenarios encompassing four datasets. Experimental results demonstrate that our proposed POND model outperforms all state-of-the-art comparison methods by up to $66\%$ on the F1-score.

POND: Multi-Source Time Series Domain Adaptation with Information-Aware Prompt Tuning

TL;DR

PrOmpt-based domaiN Discrimination (POND) is introduced, the first framework to utilize prompts for time series domain adaptation and two criteria to select good prompts are proposed, which are used to choose the most suitable source domain for domain adaptation.

Abstract

Time series domain adaptation stands as a pivotal and intricate challenge with diverse applications, including but not limited to human activity recognition, sleep stage classification, and machine fault diagnosis. Despite the numerous domain adaptation techniques proposed to tackle this complex problem, they primarily focus on domain adaptation from a single source domain. Yet, it is more crucial to investigate domain adaptation from multiple domains due to the potential for greater improvements. To address this, three important challenges need to be overcome: 1). The lack of exploration to utilize domain-specific information for domain adaptation, 2). The difficulty to learn domain-specific information that changes over time, and 3). The difficulty to evaluate learned domain-specific information. In order to tackle these challenges simultaneously, in this paper, we introduce PrOmpt-based domaiN Discrimination (POND), the first framework to utilize prompts for time series domain adaptation. Specifically, to address Challenge 1, we extend the idea of prompt tuning to time series analysis and learn prompts to capture common and domain-specific information from all source domains. To handle Challenge 2, we introduce a conditional module for each source domain to generate prompts from time series input data. For Challenge 3, we propose two criteria to select good prompts, which are used to choose the most suitable source domain for domain adaptation. The efficacy and robustness of our proposed POND model are extensively validated through experiments across 50 scenarios encompassing four datasets. Experimental results demonstrate that our proposed POND model outperforms all state-of-the-art comparison methods by up to on the F1-score.
Paper Structure (18 sections, 5 theorems, 15 equations, 5 figures, 5 tables, 1 algorithm)

This paper contains 18 sections, 5 theorems, 15 equations, 5 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setup
  4. Prompt-based Domain Discrimination
  5. The Flexible Prompt Generator
  6. Two Important Criteria for Good Prompts
  7. The Learning Objective
  8. Discussion
  9. Model Architecture and Implementation
  10. Theoretical Analysis
  11. Comparison and Relation with Previous Methods
  12. Experiments
  13. Experimental Settings
  14. Experimental Results
  15. Conclusion
  16. ...and 3 more sections

Key Result

Theorem 1

Let $1\leq q<\infty$ and $\varepsilon>0$, and $\mathcal{F}^{(S_i)}: [0,1]^{n\times L}\rightarrow [0,1]^{\vert C\vert}$ is a time series classifer, which is trained from source domain $S_i$ and is $\mathcal{L}$-Lipschitz, there exist a prompt length $m$ and a POND model $f$ such that for any $\mathca

Figures (5)

  • Figure 1: Pipeline of our proposed POND model: Step 1 pretrains the proposed POND model; Step 2 learns prompts of all source domains and the target domain; Step 3 utilizes learned prompts to select the most similar source domain to the target domain for domain adaptation.
  • Figure 2: Illustration of two criteria: high fidelity and high distinction. Fidelity and distinction are represented as areas of $A+B$ and $C$, respectively.
  • Figure 3: The F1-score and accuracy of all methods on four benchmark datasets: the proposed POND outperforms comparison methods consistently.
  • Figure 4: The F1-score and accuracy of the proposed POND model with different source domains: the performance grows with the increase of source domains. (The HHAR dataset has less than 10 domains.)
  • Figure 5: The visualization of the exponent of discrimination loss: most pairs of source domains are well discriminated.

Theorems & Definitions (7)

  • Theorem 1: Universality of our POND Model
  • Theorem 2: Flexibility of of our POND Model
  • Definition 1: Attention Layer
  • Definition 2: Simplified POND Model
  • Theorem 1: Universality of our POND Model
  • Lemma 1
  • Theorem 2: Flexibility of of our POND Model