Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TSPulse: Tiny Pre-Trained Models with Disentangled Representations for Rapid Time-Series Analysis

Vijay Ekambaram, Subodh Kumar, Arindam Jati, Sumanta Mukherjee, Tomoya Sakai, Pankaj Dayama, Wesley M. Gifford, Jayant Kalagnanam

TL;DR

TSPulse introduces a novel pre-training framework that augments masked reconstruction with explicit disentanglement across spaces and abstractions, learning three complementary embedding views (temporal, spectral, and semantic) to effectively enable zero-shot transfer.

Abstract

Time-series tasks often benefit from signals expressed across multiple representation spaces (e.g., time vs. frequency) and at varying abstraction levels (e.g., local patterns vs. global semantics). However, existing pre-trained time-series models entangle these heterogeneous signals into a single large embedding, limiting transferability and direct zero-shot usability. To address this, we propose TSPulse, family of ultra-light pre-trained models (1M parameters) with disentanglement properties, specialized for various time-series diagnostic tasks. TSPulse introduces a novel pre-training framework that augments masked reconstruction with explicit disentanglement across spaces and abstractions, learning three complementary embedding views (temporal, spectral, and semantic) to effectively enable zero-shot transfer. In-addition, we introduce various lightweight post-hoc fusers that selectively attend and fuse these disentangled views based on task type, enabling simple but effective task specializations. To further improve robustness and mitigate mask-induced bias prevalent in existing approaches, we propose a simple yet effective hybrid masking strategy that enhances missing diversity during pre-training. Despite its compact size, TSPulse achieves strong and consistent gains across four TS diagnostic tasks: +20% on the TSB-AD anomaly detection leaderboard, +25% on similarity search, +50% on imputation, and +5-16% on multivariate classification, outperforming models that are 10-100X larger on over 75 datasets. TSPulse delivers state-of-the-art zero-shot performance, efficient fine-tuning, and supports GPU-free deployment. Models and source code are publicly available at https://huggingface.co/ibm-granite/granite-timeseries-tspulse-r1.

TSPulse: Tiny Pre-Trained Models with Disentangled Representations for Rapid Time-Series Analysis

TL;DR

TSPulse introduces a novel pre-training framework that augments masked reconstruction with explicit disentanglement across spaces and abstractions, learning three complementary embedding views (temporal, spectral, and semantic) to effectively enable zero-shot transfer.

Abstract

Time-series tasks often benefit from signals expressed across multiple representation spaces (e.g., time vs. frequency) and at varying abstraction levels (e.g., local patterns vs. global semantics). However, existing pre-trained time-series models entangle these heterogeneous signals into a single large embedding, limiting transferability and direct zero-shot usability. To address this, we propose TSPulse, family of ultra-light pre-trained models (1M parameters) with disentanglement properties, specialized for various time-series diagnostic tasks. TSPulse introduces a novel pre-training framework that augments masked reconstruction with explicit disentanglement across spaces and abstractions, learning three complementary embedding views (temporal, spectral, and semantic) to effectively enable zero-shot transfer. In-addition, we introduce various lightweight post-hoc fusers that selectively attend and fuse these disentangled views based on task type, enabling simple but effective task specializations. To further improve robustness and mitigate mask-induced bias prevalent in existing approaches, we propose a simple yet effective hybrid masking strategy that enhances missing diversity during pre-training. Despite its compact size, TSPulse achieves strong and consistent gains across four TS diagnostic tasks: +20% on the TSB-AD anomaly detection leaderboard, +25% on similarity search, +50% on imputation, and +5-16% on multivariate classification, outperforming models that are 10-100X larger on over 75 datasets. TSPulse delivers state-of-the-art zero-shot performance, efficient fine-tuning, and supports GPU-free deployment. Models and source code are publicly available at https://huggingface.co/ibm-granite/granite-timeseries-tspulse-r1.
Paper Structure (88 sections, 5 equations, 14 figures, 37 tables)

This paper contains 88 sections, 5 equations, 14 figures, 37 tables.

Table of Contents

  1. Introduction
  2. TSPulse Architecture
  3. TSPulse Workflows
  4. Pre-training:
  5. Target data Fine-Tuning:
  6. Downstream Tasks & Post-Hoc Fusers:
  7. Experiments
  8. Anomaly Detection (AD)
  9. Classification
  10. Imputation
  11. Time-Series Similarity Search
  12. Ablation studies
  13. Sensitivity Analysis of Embedding Disentanglement
  14. Conclusion
  15. Appendix
  16. ...and 73 more sections

Figures (14)

  • Figure 1: TSPulse Overview.% and X represent the accuracy and size improvements of TSPulse over SOTA pre-trained models across different benchmarks.
  • Figure 2: TSPulse Architecture. (X: Inputs, Y: Outputs, E: Embeddings). Annotations for easy reference: ①- ⑥.
  • Figure 3: TSPulse Downstream Differentiators.
  • Figure 4: VUS-PR (official metric, higher is better) scores of different AD methods on the TSB-AD leaderboard; IMP(%)—the percentage improvement of TSPulse over baselines; A maximum of top three SOTA models have been reported in each category. Full results across all 40 models are in Appendix \ref{['app:anomaly']}.
  • Figure 5: Classification Mean Accuracy results (higher is better); IMP(%)—the percentage improvement of TSPulse over baselines.
  • ...and 9 more figures