Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prompting-based Temporal Domain Generalization

Sepidehsadat Hosseini, Mengyao Zhai, Hossein Hajimirsadegh, Frederick Tung

TL;DR

The paper tackles temporal distribution shifts by introducing a prompting-based temporal domain generalization framework that keeps a frozen backbone while learning compact prompts. It decomposes prompts into domain-specific, temporal, and global components, with a temporal prompt generator forecasting future prompts from past domain prompts. The approach shows strong, parameter-efficient generalization across classification, regression, and time-series forecasting, outperforming state-of-the-art temporal DG methods on synthetic and real datasets while reducing training and inference overhead. This enables robust predictions in evolving environments without access to future data during training, with broad applicability to modern transformer-based architectures. The results suggest prompting-based temporal DG as a practical strategy for real-world deployment under non-i.i.d. temporal drift.

Abstract

Machine learning traditionally assumes that the training and testing data are distributed independently and identically. However, in many real-world settings, the data distribution can shift over time, leading to poor generalization of trained models in future time periods. This paper presents a novel prompting-based approach to temporal domain generalization that is parameter-efficient, time-efficient, and does not require access to future data during training. Our method adapts a trained model to temporal drift by learning global prompts, domain-specific prompts, and drift-aware prompts that capture underlying temporal dynamics. Experiments on classification, regression, and time series forecasting tasks demonstrate the generality of the proposed approach. The code repository will be publicly shared.

Prompting-based Temporal Domain Generalization

TL;DR

The paper tackles temporal distribution shifts by introducing a prompting-based temporal domain generalization framework that keeps a frozen backbone while learning compact prompts. It decomposes prompts into domain-specific, temporal, and global components, with a temporal prompt generator forecasting future prompts from past domain prompts. The approach shows strong, parameter-efficient generalization across classification, regression, and time-series forecasting, outperforming state-of-the-art temporal DG methods on synthetic and real datasets while reducing training and inference overhead. This enables robust predictions in evolving environments without access to future data during training, with broad applicability to modern transformer-based architectures. The results suggest prompting-based temporal DG as a practical strategy for real-world deployment under non-i.i.d. temporal drift.

Abstract

Machine learning traditionally assumes that the training and testing data are distributed independently and identically. However, in many real-world settings, the data distribution can shift over time, leading to poor generalization of trained models in future time periods. This paper presents a novel prompting-based approach to temporal domain generalization that is parameter-efficient, time-efficient, and does not require access to future data during training. Our method adapts a trained model to temporal drift by learning global prompts, domain-specific prompts, and drift-aware prompts that capture underlying temporal dynamics. Experiments on classification, regression, and time series forecasting tasks demonstrate the generality of the proposed approach. The code repository will be publicly shared.
Paper Structure (21 sections, 4 equations, 6 figures, 9 tables)

This paper contains 21 sections, 4 equations, 6 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Backbone Network Pre-Training
  5. Domain-specific Prompt Learning
  6. Temporal Prompt Learning
  7. Inference
  8. Experiments
  9. Baseline Methods
  10. Implementation Details
  11. Results on Synthetic Data
  12. Main Experimental Results
  13. Ablation Studies and Additional Experiments
  14. CONCLUSION
  15. Appendix
  16. ...and 6 more sections

Figures (6)

  • Figure 1: Qualitative results on Sum of Cosines synthetic dataset generated with phase-frequency modification and addition of a variable cosine wave.
  • Figure 2: Pairwise comparisons of learned temporal prompts ($P_Ts$) across domains in the Sum of Cosines synthetic dataset with both types of synthetic drift. The values shown are cosine similarities.
  • Figure 3: Applying temporal shift to Mackey-Glass time series by modifying $\sigma$.
  • Figure 4: Applying temporal shift to Mackey-Glass time series by adding variable cosine wave.
  • Figure 5: Applying temporal shift to Sum of Cosines time series by modifying phase and frequency.
  • ...and 1 more figures