Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective

TL;DR

Detailed theoretical analysis and abundant empirical evidence consistently show that Attraos outperforms various LTSF methods on mainstream LTSF datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.

Abstract

In long-term time series forecasting (LTSF) tasks, an increasing number of models have acknowledged that discrete time series originate from continuous dynamic systems and have attempted to model their dynamical structures. Recognizing the chaotic nature of real-world data, our model, \textbf{\textit{Attraos}}, incorporates chaos theory into LTSF, perceiving real-world time series as observations from unknown high-dimensional chaotic dynamic systems. Under the concept of attractor invariance, Attraos utilizes non-parametric Phase Space Reconstruction embedding and the proposed multi-scale dynamic memory unit to memorize historical dynamics structure and predicts by a frequency-enhanced local evolution strategy. Detailed theoretical analysis and abundant empirical evidence consistently show that Attraos outperforms various LTSF methods on mainstream LTSF datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.

Figures (8)

• Figure 1: (a): Classical chaotic systems with noise. (b): dynamical system structure of real-world datasets. (c): Different types of Attractors. See more figures in Appendix \ref{['more psr fig']}.
• Figure 2: Overall architecture of Attraos. Initially, the PSR technique is employed to restore the underlying dynamical structures from historical data $\{z_i\}$. Subsequently, the dynamical system trajectory is fed into MDMU, projected onto polynomial space $\mathcal{G}_\theta^N$ using a time window $\theta$ and polynomial order $N$. Gradually, a hierarchical projection is performed to obtain more macroscopic memories of the dynamical system structure. Finally, local evolution operator $\mathcal{K}^{(i)}$ in the frequency domain is employed to obtain future state, thereby for the prediction.
• Figure 3: (a) Discretization of continuous polynomial approximation for sequence data. $g$ represents the optimal polynomial constructed from polynomial bases. (b) MDMU projects the dynamical structure onto different orthogonal subspaces $\mathcal{G}$ and $\mathcal{S}$. (c) Sequential computation for Eq. \ref{['eq:ssm-a']} in $\mathcal{O}(L)$ time complexity. (d) Blelloch tree scanning for Eq. \ref{['eq:ssm-a']} in $\mathcal{O}(logL)$ by storing intermediate results.
• Figure 4: Complexity analysis.
• Figure 5: Left: Chaotic Reconstruction for Lorenz96 system with 720 forecasting step. Right: Hyper-parameter analysis w.r.t. polynomial orders for different model variants w.r.t. patching operation in ETTm2 dataset.

Theorems & Definitions (21)

• Remark 3.1
• Remark 3.2
• Proposition 1
• Remark 3.3
• Remark 3.4
• Remark 3.5
• Theorem 2
• Remark 3.6
• Theorem 3
• Theorem 4