Long Range Switching Time Series Prediction via State Space Model

TL;DR

The paper tackles the challenge of long-range dependencies in switching time series by fusing Structured State Space Models (S4) with Switching Non-linear Dynamics (SNLDS). It introduces two innovations: the offline S4 split, which segments irregular data using change-point detection and retrains S4 on each segment, and the online S4 SNLDS, a hybrid that leverages S4 memory within a Bayesian switching framework for robust segmentation and mode-aware generation. Empirical results on 1-D Lorenz dynamics and 2-D bouncing ball data show improved reconstruction and long-horizon prediction compared with baselines (S4 and SNLDS), with notable gains in forecasting range and segmentation accuracy. The work also demonstrates substantial speedups through FlashConv and Vandermonde-based kernel computations, suggesting practical benefits for long sequence modeling in fields like neuroscience and medical time series.

Abstract

In this study, we delve into the Structured State Space Model (S4), Change Point Detection methodologies, and the Switching Non-linear Dynamics System (SNLDS). Our central proposition is an enhanced inference technique and long-range dependency method for SNLDS. The cornerstone of our approach is the fusion of S4 and SNLDS, leveraging the strengths of both models to effectively address the intricacies of long-range dependencies in switching time series. Through rigorous testing, we demonstrate that our proposed methodology adeptly segments and reproduces long-range dependencies in both the 1-D Lorenz dataset and the 2-D bouncing ball dataset. Notably, our integrated approach outperforms the standalone SNLDS in these tasks.

Figures (14)

• Figure 1: Illustration of the generative model.
• Figure 2: Flow chart of S4 split.
• Figure 3: Result of change point detection.
• Figure 4: Structure of S4 model.
• Figure 5: Reconstruction on 1-D switching time series.