Time-SSM: Simplifying and Unifying State Space Models for Time Series Forecasting

TL;DR

This work addresses the gap in principled, scalable time-series forecasting with State Space Models by introducing Dynamic Spectral Operator theory and Time-SSM. It unifies SSMs with spectral-transform concepts, leverages HiPPO-LegP and diagonal/complex-plane variants, and implements a lightweight Time-SSM foundation that achieves strong long-horizon forecasting with far fewer parameters than prior black-box SSMs. Through extensive ablations and cross-dataset experiments, the authors demonstrate that time-varying spectral representations, unitary biases, and patch-based embeddings drive performance and efficiency. The proposed framework offers a theoretically grounded, practical path to applying SSMs to TSF tasks and highlights future directions for multi-scale, multi-variate extensions.

Abstract

State Space Models (SSMs) have emerged as a potent tool in sequence modeling tasks in recent years. These models approximate continuous systems using a set of basis functions and discretize them to handle input data, making them well-suited for modeling time series data collected at specific frequencies from continuous systems. Despite its potential, the application of SSMs in time series forecasting remains underexplored, with most existing models treating SSMs as a black box for capturing temporal or channel dependencies. To address this gap, this paper proposes a novel theoretical framework termed Dynamic Spectral Operator, offering more intuitive and general guidance on applying SSMs to time series data. Building upon our theory, we introduce Time-SSM, a novel SSM-based foundation model with only one-seventh of the parameters compared to Mamba. Various experiments validate both our theoretical framework and the superior performance of Time-SSM.

Figures (7)

• Figure 1: From generalized orthogonal basis projection theory to dynamic spectral operator theory. (a) In the TSF task, we aim to find a complex nonlinear mapping $\mathcal{K}$ from past observed data $u$ to future predicted data $y$. (b) Neural spectral operators simplify the parameterization of mapping $\mathcal{K}$ by leveraging an easily learnable spectral space. (c) The GOBP theory represents the projection and reversible reconstruction of input data $u$ to the spectral space $x$. (d) We define the learnable SSMs as a dynamic operator that gradually shifts from an initial spectral projection space toward an optimal spectral space and find a linear projection to the observation $y$.
• Figure 2: Classical spectral transform with basis function and inner product measure.
• Figure 3: Architecture of Time-SSM, $\bm{PQ}$ represent vectorized representation of a function.
• Figure 4: Left: Complexity analysis. Right: Long-range function approximation with different SSM basis
• Figure 5: hu2024attractor Elaboration of Hippo-LegP

Theorems & Definitions (35)

• Lemma 2.1
• Lemma 2.2
• Definition 1
• Remark 2.3
• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4
• Lemma 2.4
• Corollary 2.5