Meta-learning to Address Data Shift in Time Series Classification

TL;DR

Data shift causes traditional time-series models to degrade when $p_{\mathrm{test}}(x,y) \neq p_{\mathrm{train}}(x,y)$. The authors evaluate meta-learning (Reptile, FOMAML) against TD learning and a divide-and-conquer baseline on SeisTask and OOD-STEAD to study fast adaptation with few-shot data. They find meta-learning offers faster, more stable adaptation in data-scarce regimes and small models, with advantages diminishing as data and capacity grow, and show that distribution alignment between training and test tasks, not mere task diversity, drives gains. The SeisTask benchmark is introduced as a controlled, domain-relevant time-series dataset to advance adaptive learning in physics-based domains.

Abstract

Across engineering and scientific domains, traditional deep learning (TDL) models perform well when training and test data share the same distribution. However, the dynamic nature of real-world data, broadly termed \textit{data shift}, renders TDL models prone to rapid performance degradation, requiring costly relabeling and inefficient retraining. Meta-learning, which enables models to adapt quickly to new data with few examples, offers a promising alternative for mitigating these challenges. Here, we systematically compare TDL with fine-tuning and optimization-based meta-learning algorithms to assess their ability to address data shift in time-series classification. We introduce a controlled, task-oriented seismic benchmark (SeisTask) and show that meta-learning typically achieves faster and more stable adaptation with reduced overfitting in data-scarce regimes and smaller model architectures. As data availability and model capacity increase, its advantages diminish, with TDL with fine-tuning performing comparably. Finally, we examine how task diversity influences meta-learning and find that alignment between training and test distributions, rather than diversity alone, drives performance gains. Overall, this work provides a systematic evaluation of when and why meta-learning outperforms TDL under data shift and contributes SeisTask as a benchmark for advancing adaptive learning research in time-series domains.

Figures (16)

• Figure 1: Three replicates of signal waveforms (synthetic signals embedded in real noise) are shown for tasks 0 (top) and 143 (bottom). In the bottom right panel, signal is seen occurring just before 2s, but identifying whether a signal occurs is usually challenging (e.g., in the bottom left panel). Task 0 represents the simulator set at Circles:0, Layers:0, Velocity:Lo, Frequency:Lo, Source:Ricker. Task 143 represents the simulator set at Circles:2, Layers:4, Velocity:Lo, Frequency:Hi, and Source:Gabor. See Appendix \ref{['sec:appendix_data']} for more details.
• Figure 2: Empirical distribution of the proportion $p_u$ across all tasks in SeisTask. The distribution’s concentration near 1 indicates that models capture task-specific features that improve within-task performance, while its spread reflects transferable information between tasks.
• Figure 3: Relationship between cross-task accuracy and task similarity. Each point represents paired tasks binned by similarity. Cross-task accuracy in each bin is standardized to unit variance for display. The positive trend indicates that more similar tasks have better cross-wise performance (when a task-specific model is trained on one task and evaluated on another, and vice-versa), validating similarity as a proxy for relevant data shift.
• Figure 4: The dendrogram resulting from agglomerative clustering based on task similarity in SeisTask. The tasks on the left side are assigned to the test split, the rest are assigned to the training and validation split. Train A and Train B are distinguished for their relationships to the test split, where Train A is less similar to the test split than Train B.
• Figure 5: Average accuracy of task-specific models when trained on tasks within a group and evaluated on tasks in other groups. Performance decreases substantially when models trained on Train A or Train B tasks are evaluated on the test tasks, confirming the presence of relevant data shift between the splits.