SwiftTS: A Swift Selection Framework for Time Series Pre-trained Models via Multi-task Meta-Learning

TL;DR

SwiftTS tackles the challenge of selecting the best time series pre-trained model from a large model hub without expensive fine-tuning. It introduces a dual-encoder framework that computes patchwise data-model compatibility under horizon-aware conditions, augmented by horizon-adaptive experts and transferable cross-task meta-learning to generalize across datasets and horizons. The approach yields state-of-the-art ranking performance (measured by $\tau_\omega$) while substantially reducing computation compared to brute-force fine-tuning, demonstrated across 14 datasets and 8 models. This enables practical, scalable deployment of time series pre-trained models with reliable cross-domain performance predictions.

Abstract

Pre-trained models exhibit strong generalization to various downstream tasks. However, given the numerous models available in the model hub, identifying the most suitable one by individually fine-tuning is time-consuming. In this paper, we propose \textbf{SwiftTS}, a swift selection framework for time series pre-trained models. To avoid expensive forward propagation through all candidates, SwiftTS adopts a learning-guided approach that leverages historical dataset-model performance pairs across diverse horizons to predict model performance on unseen datasets. It employs a lightweight dual-encoder architecture that embeds time series and candidate models with rich characteristics, computing patchwise compatibility scores between data and model embeddings for efficient selection. To further enhance the generalization across datasets and horizons, we introduce a horizon-adaptive expert composition module that dynamically adjusts expert weights, and the transferable cross-task learning with cross-dataset and cross-horizon task sampling to enhance out-of-distribution (OOD) robustness. Extensive experiments on 14 downstream datasets and 8 pre-trained models demonstrate that SwiftTS achieves state-of-the-art performance in time series pre-trained model selection.

Figures (6)

• Figure 1: SwiftTS employs an efficient learning-guided selection framework for time series forecasting, enabling horizon-specific selection and improved cross-domain generalization via multi-task meta-learning.
• Figure 2: The framework of SwiftTS, consisting of (1) a temporal-aware data encoder, (2) a knowledge-infused model encoder, (3) patchwise cross-attention, (4) a horizon-adaptive expert composition module, and (5) the transferable cross-task learning.
• Figure 3: Parameter update process in cross-task learning (top-left), and sampling strategies for cross-horizon and cross-dataset tasks (bottom).
• Figure 4: (a) Comparison of (a) average $\tau_\omega$ for IID vs. OOD settings across four datasets and horizons, and (b) ablation study of cross-task learning, and method comparison w.r.t running time (second) and average $\tau_\omega$ across horizons on (c) ETTh1 (small-scale) and (d) Traffic (large-scale).
• Figure 5: (a) Sensitivity analysis of the loss coefficient $\lambda$, (b) choice of the number of experts $G$.