TS-ACL: Closed-Form Solution for Time Series-oriented Continual Learning

TL;DR

TS-ACL tackles time-series class-incremental learning by replacing gradient-based updates with a gradient-free, closed-form ridge regression framework built on a frozen encoder and a multi-scale feature fusion pipeline. It achieves stability by avoiding gradient interference across tasks and accomplishes plasticity through global distribution learning via recursive updates and high-dimensional mappings, enhanced by a Randomly-initialized Hidden Layer ensemble. The approach is exemplar-free and privacy-preserving, with theoretical guarantees showing equivalence to joint training and strong empirical SOTA performance on five benchmarks, while offering much higher efficiency suitable for edge and real-time systems. Overall, TS-ACL delivers a scalable, privacy-conscious continual learning solution for streaming time-series data with robust stability-plasticity trade-offs and practical applicability.

Abstract

Time series classification underpins critical applications such as healthcare diagnostics and gesture-driven interactive systems in multimedia scenarios. However, time series class-incremental learning (TSCIL) faces two major challenges: catastrophic forgetting and intra-class variations. Catastrophic forgetting occurs because gradient-based parameter update strategies inevitably erase past knowledge. And unlike images, time series data exhibits subject-specific patterns, also known as intra-class variations, which refer to differences in patterns observed within the same class. While exemplar-based methods fail to cover diverse variation with limited samples, existing exemplar-free methods lack explicit mechanisms to handle intra-class variations. To address these two challenges, we propose TS-ACL, which leverages a gradient-free closed-form solution to avoid the catastrophic forgetting problem inherent in gradient-based optimization methods while simultaneously learning global distributions to resolve intra-class variations. Additionally, it provides privacy protection and efficiency. Extensive experiments on five benchmark datasets covering various sensor modalities and tasks demonstrate that TS-ACL achieves performance close to joint training on four datasets, outperforming existing methods and establishing a new state-of-the-art (SOTA) for TSCIL.

Figures (5)

• Figure 1: Schematic diagram of Time Series Class-Incremental Learning (TSCIL) and a brief TS-ACL model update illustration. As the data stream continuously arrives, the model learns new classes in each task.
• Figure 2: Overview of the proposed TS-ACL. In (a), a pre-trained encoder is first obtained on Task 1 through Gradient Descent-Based Training. Next, as shown in (b), a Regression-Based Coordinate module with an RHL is introduced before the classification head to enhance the feature dimension, resulting in $\mathbf{\Psi_1}$ from Task 1 data. Finally, in (c), a Recursive Regression-Based Learning process is applied across tasks.
• Figure 3: t-SNE visualization of DSA dataset features. Different colors represent different classes, and it can be clearly seen that a single class contains many sub-clusters.
• Figure 4: The accuracy of all methods on five datasets changes as the tasks progress. "Naive" refers to the baseline method without any incremental learning strategies, while "Offline" indicates methods that are trained on the entire dataset at once.
• Figure 5: Running time comparison (in seconds) across different methods on five benchmark datasets. TS-ACL consistently achieves the fastest training time across all datasets, demonstrating its superior computational efficiency. This significant reduction in computational time is attributed to TS-ACL's analytical learning approach, which requires only a single update per task after the initial training.

Theorems & Definitions (4)

• Remark 1
• Remark 2
• Theorem 1
• proof