Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A SSM is Polymerized from Multivariate Time Series

Haixiang Wu

TL;DR

This work delves into the derivation of SSM, which involves approximating continuously updated functions by orthogonal function basis, and develops Poly-Mamba, a novel method for MTS forecasting.

Abstract

For multivariate time series (MTS) tasks, previous state space models (SSMs) followed the modeling paradigm of Transformer-based methods. However, none of them explicitly model the complex dependencies of MTS: the Channel Dependency variations with Time (CDT). In view of this, we delve into the derivation of SSM, which involves approximating continuously updated functions by orthogonal function basis. We then develop Poly-Mamba, a novel method for MTS forecasting. Its core concept is to expand the original orthogonal function basis space into a multivariate orthogonal function space containing variable mixing terms, and make a projection on this space so as to explicitly describe the CDT by weighted coefficients. In Poly-Mamba, we propose the Multivariate Orthogonal Polynomial Approximation (MOPA) as a simplified implementation of this concept. For the simple linear relationship between channels, we propose Linear Channel Mixing (LCM) and generate CDT patterns adaptively for different channels through a proposed Order Combining method. Experiments on six real-world datasets demonstrate that Poly-Mamba outperforms the SOTA methods, especially when dealing with datasets having a large number of channels and complex correlations. The codes and log files will be released at: https://github.com/Joeland4/Poly-Mamba.

A SSM is Polymerized from Multivariate Time Series

TL;DR

This work delves into the derivation of SSM, which involves approximating continuously updated functions by orthogonal function basis, and develops Poly-Mamba, a novel method for MTS forecasting.

Abstract

For multivariate time series (MTS) tasks, previous state space models (SSMs) followed the modeling paradigm of Transformer-based methods. However, none of them explicitly model the complex dependencies of MTS: the Channel Dependency variations with Time (CDT). In view of this, we delve into the derivation of SSM, which involves approximating continuously updated functions by orthogonal function basis. We then develop Poly-Mamba, a novel method for MTS forecasting. Its core concept is to expand the original orthogonal function basis space into a multivariate orthogonal function space containing variable mixing terms, and make a projection on this space so as to explicitly describe the CDT by weighted coefficients. In Poly-Mamba, we propose the Multivariate Orthogonal Polynomial Approximation (MOPA) as a simplified implementation of this concept. For the simple linear relationship between channels, we propose Linear Channel Mixing (LCM) and generate CDT patterns adaptively for different channels through a proposed Order Combining method. Experiments on six real-world datasets demonstrate that Poly-Mamba outperforms the SOTA methods, especially when dealing with datasets having a large number of channels and complex correlations. The codes and log files will be released at: https://github.com/Joeland4/Poly-Mamba.
Paper Structure (14 sections, 13 equations, 4 figures, 3 tables)

This paper contains 14 sections, 13 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. SSM
  4. Multivariable Legendre polynomials
  5. Poly-Mamba
  6. MOPA
  7. LCM
  8. Order Combining
  9. Experiments
  10. Main Results
  11. Ablations
  12. Visual Analysis
  13. Efficiency
  14. Conclusion

Figures (4)

  • Figure 1: (left) Poly-Mamba architecture. (right) Three operations on coefficients in the improved SSM.
  • Figure 2: (left) One segment of normalized input of Weather dataset. (right) LCM weight matrix L, which is averaged on all encoder layers.
  • Figure 3: (left) The first normalized test input of ETTh2 dataset. (middle) The corresponding hidden state value, averaged on all encoder layers, before MOPA operation. (right) The corresponding hidden state value after MOPA operation.
  • Figure 4: The forecasting of the first test time series from the ECL dataset.