Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Zero-Shot Time Series Classification: From Task-specific Classifiers to In-Context Inference

Juntao Fang, Shifeng Xie, Shengbin Nie, Yuhui Ling, Yuming Liu, Zijian Li, Keli Zhang, Lujia Pan, Themis Palpanas, Ruichu Cai

TL;DR

This work challenges the standard zero-shot evaluation of time-series foundation models that relies on a frozen encoder plus a task-specific classifier. It introduces TIC-FM, a training-free in-context learning framework that conditions on the labeled support set and predicts all test labels in a single forward pass, using a ViT-based encoder, a projection adapter, and a split-masked latent-memory in-context classifier. The authors provide theoretical justification showing that in-context inference can subsume training-based classifiers and can emulate gradient-based updates within forward computation. Empirically, TIC-FM achieves state-of-the-art performance across 128 UCR datasets, with pronounced advantages under extreme label scarcity and robust transfer without training data from the target tasks.

Abstract

The zero-shot evaluation of time series foundation models (TSFMs) for classification typically uses a frozen encoder followed by a task-specific classifier. However, this practice violates the training-free premise of zero-shot deployment and introduces evaluation bias due to classifier-dependent training choices. To address this issue, we propose TIC-FM, an in-context learning framework that treats the labeled training set as context and predicts labels for all test instances in a single forward pass, without parameter updates. TIC-FM pairs a time series encoder and a lightweight projection adapter with a split-masked latent memory Transformer. We further provide theoretical justification that in-context inference can subsume trained classifiers and can emulate gradient-based classifier training within a single forward pass. Experiments on 128 UCR datasets show strong accuracy, with consistent gains in the extreme low-label situation, highlighting training-free transfer

Rethinking Zero-Shot Time Series Classification: From Task-specific Classifiers to In-Context Inference

TL;DR

This work challenges the standard zero-shot evaluation of time-series foundation models that relies on a frozen encoder plus a task-specific classifier. It introduces TIC-FM, a training-free in-context learning framework that conditions on the labeled support set and predicts all test labels in a single forward pass, using a ViT-based encoder, a projection adapter, and a split-masked latent-memory in-context classifier. The authors provide theoretical justification showing that in-context inference can subsume training-based classifiers and can emulate gradient-based updates within forward computation. Empirically, TIC-FM achieves state-of-the-art performance across 128 UCR datasets, with pronounced advantages under extreme label scarcity and robust transfer without training data from the target tasks.

Abstract

The zero-shot evaluation of time series foundation models (TSFMs) for classification typically uses a frozen encoder followed by a task-specific classifier. However, this practice violates the training-free premise of zero-shot deployment and introduces evaluation bias due to classifier-dependent training choices. To address this issue, we propose TIC-FM, an in-context learning framework that treats the labeled training set as context and predicts labels for all test instances in a single forward pass, without parameter updates. TIC-FM pairs a time series encoder and a lightweight projection adapter with a split-masked latent memory Transformer. We further provide theoretical justification that in-context inference can subsume trained classifiers and can emulate gradient-based classifier training within a single forward pass. Experiments on 128 UCR datasets show strong accuracy, with consistent gains in the extreme low-label situation, highlighting training-free transfer
Paper Structure (80 sections, 6 theorems, 44 equations, 5 figures, 5 tables, 1 algorithm)

This paper contains 80 sections, 6 theorems, 44 equations, 5 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. TIC-FM Methodology
  5. Embedding-to-ICL Framework
  6. Time Series Feature Encoder
  7. Projection Adapter
  8. In-Context Classifier
  9. Pretraining and Inference
  10. Why In-Context Inference Works
  11. ICL Contains Trained Classifiers
  12. TIC-FM Emulates Gradient-Based Classifier Training
  13. Experiments
  14. Experimental setup
  15. Comparative Analysis on Standard UCR
  16. ...and 65 more sections

Key Result

Proposition 5.2

Let $\mathsf{Train}$ be any classifier-training procedure that maps a training set to a classifier $h_\tau:\mathbb{R}^q\to\mathbb{R}^K$. Let the corresponding score map be $F(\mathcal{D}_{\mathrm{tr}},\{z_j^{\mathrm{te}}\}_{j=1}^{n_{\mathrm{te}}}) \triangleq \{h_\tau(z_j^{\mathrm{te}})\}_{j=1}^{n_{\

Figures (5)

  • Figure 1: An overview of TIC-FM architecture. Each time series is first encoded by a ViT-based feature encoder into an instance embedding, and then mapped by a lightweight projection adapter to the token space of the in-context classifier. The classifier processes all context and query samples jointly: it consolidates long contexts via perceiver latent memory, injects label embeddings only into the context slice, and performs split-masked Transformer reasoning.
  • Figure 2: Scalability analysis with labeled data fractions.
  • Figure 3: Impact of context length on inference accuracy. Increasing the number of context examples ($N_{ctx}$) consistently improves performance.
  • Figure 4: Ablation study on UCR. Removing the in-context classifier results in a marked decrease in performance.
  • Figure 7: Extended scalability analysis with varying labeled data fractions. This figure complements Figure \ref{['fig:train_fraction_scaling']} by illustrating the performance of TIC-FM against the full set of baseline configurations, including parametric (e.g., MLP) and non-parametric (e.g., KNN, NC) classifiers omitted from the main text. Observations: TIC-FM consistently outperforms all baseline variants across all supervision budgets.

Theorems & Definitions (10)

  • Proposition 5.2: ICL subsumes the trained-classifier zero-shot pipeline
  • Proposition 5.3: In-context emulation of gradient descent
  • Lemma 2.1: Density of polynomials symmetric in training blocks
  • proof
  • Lemma 2.2: Masked pooling of training features
  • proof
  • Lemma 2.3: Broadcast and apply a continuous map
  • proof
  • proof : Completion of the proof
  • Corollary 2.4: Label matching under a uniform margin