LiNo: Advancing Recursive Residual Decomposition of Linear and Nonlinear Patterns for Robust Time Series Forecasting

TL;DR

The proposed LiNo achieves state-of-the-art on thirteen real-world benchmarks under univariate and multivariate forecasting scenarios, and experiments show that current forecasting models can deliver more robust and precise results through this advanced Recursive Residual Decomposition.

Abstract

Forecasting models are pivotal in a data-driven world with vast volumes of time series data that appear as a compound of vast Linear and Nonlinear patterns. Recent deep time series forecasting models struggle to utilize seasonal and trend decomposition to separate the entangled components. Such a strategy only explicitly extracts simple linear patterns like trends, leaving the other linear modes and vast unexplored nonlinear patterns to the residual. Their flawed linear and nonlinear feature extraction models and shallow-level decomposition limit their adaptation to the diverse patterns present in real-world scenarios. Given this, we innovate Recursive Residual Decomposition by introducing explicit extraction of both linear and nonlinear patterns. This deeper-level decomposition framework, which is named LiNo, captures linear patterns using a Li block which can be a moving average kernel, and models nonlinear patterns using a No block which can be a Transformer encoder. The extraction of these two patterns is performed alternatively and recursively. To achieve the full potential of LiNo, we develop the current simple linear pattern extractor to a general learnable autoregressive model, and design a novel No block that can handle all essential nonlinear patterns. Remarkably, the proposed LiNo achieves state-of-the-art on thirteen real-world benchmarks under univariate and multivariate forecasting scenarios. Experiments show that current forecasting models can deliver more robust and precise results through this advanced Recursive Residual Decomposition. We hope this work could offer insight into designing more effective forecasting models. Code is available at this Repository: https://github.com/Levi-Ackman/LiNo.

Figures (30)

• Figure 1: Example of the multi-level linear and nonlinear patterns in real-world time series. We take the ETTh2 dataset as an example and decompose a raw time series (Raw) into four signals through linear and nonlinear patterns decomposition. Linear 1&2 is the obtained linear patterns using the proposed Li block, and Nonlinear 1&2 is the obtained nonlinear patterns using No block. In other words, the raw time series (red) is the sum of the four signals below.
• Figure 2: Framework of LiNo. Li block and No block extract patterns from the embedded input alternatively, in an RRD manner. The final prediction is aggregated from all blocks.
• Figure 3: Multivariate forecasting performance of three different model designs using iTransformer as backbone under different noise levels across datasets of ECL, ETTm2, and Weather.
• Figure 4: Multivariate forecasting performance improves with the increase of lookback window $T \in \{48, 96, 192, 336, 720\}$ and a fixed prediction length $F = 720$. Notably, LiNo consistently and stably enhances its forecasting performance as the lookback window size increases.
• Figure 5: Visualization of LiNo's multivariate forecasting result on ECL dataset. 'LP' denotes Linear prediction, and 'NP' stands for Nonlinear prediction. LP $i$ or NP $i$ ($i \in \{1,2,3\}$) is the linear or nonlinear prediction of $i$-th layer (level). Same to followed figures.