Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ISMRNN: An Implicitly Segmented RNN Method with Mamba for Long-Term Time Series Forecasting

GaoXiang Zhao, Li Zhou, XiaoQiang Wang

TL;DR

ISMRNN tackles long-term time series forecasting by addressing information loss and gradient issues inherent to RNNs, building on SegRNN with three enhancements: implicit segmentation for denser information exchange, a residual encoding path to bypass part of the recurrent flow, and Mamba-based preprocessing to better extract temporal structure. The method shows superior performance across six real-world datasets, achieving top results on a majority of MSE/MAE metrics and demonstrating notable gains especially at shorter look-back windows. Ablation confirms a synergistic effect among the three components, while efficiency remains competitive with non-RNN baselines. Overall, ISMRNN offers a practical, scalable approach to long-horizon forecasting with improved information flow and robustness to horizon length.

Abstract

Long time series forecasting aims to utilize historical information to forecast future states over extended horizons. Traditional RNN-based series forecasting methods struggle to effectively address long-term dependencies and gradient issues in long time series problems. Recently, SegRNN has emerged as a leading RNN-based model tailored for long-term series forecasting, demonstrating state-of-the-art performance while maintaining a streamlined architecture through innovative segmentation and parallel decoding techniques. Nevertheless, SegRNN has several limitations: its fixed segmentation disrupts data continuity and fails to effectively leverage information across different segments, the segmentation strategy employed by SegRNN does not fundamentally address the issue of information loss within the recurrent structure. To address these issues, we propose the ISMRNN method with three key enhancements: we introduce an implicit segmentation structure to decompose the time series and map it to segmented hidden states, resulting in denser information exchange during the segmentation phase. Additionally, we incorporate residual structures in the encoding layer to mitigate information loss within the recurrent structure. To extract information more effectively, we further integrate the Mamba architecture to enhance time series information extraction. Experiments on several real-world long time series forecasting datasets demonstrate that our model surpasses the performance of current state-of-the-art models.

ISMRNN: An Implicitly Segmented RNN Method with Mamba for Long-Term Time Series Forecasting

TL;DR

ISMRNN tackles long-term time series forecasting by addressing information loss and gradient issues inherent to RNNs, building on SegRNN with three enhancements: implicit segmentation for denser information exchange, a residual encoding path to bypass part of the recurrent flow, and Mamba-based preprocessing to better extract temporal structure. The method shows superior performance across six real-world datasets, achieving top results on a majority of MSE/MAE metrics and demonstrating notable gains especially at shorter look-back windows. Ablation confirms a synergistic effect among the three components, while efficiency remains competitive with non-RNN baselines. Overall, ISMRNN offers a practical, scalable approach to long-horizon forecasting with improved information flow and robustness to horizon length.

Abstract

Long time series forecasting aims to utilize historical information to forecast future states over extended horizons. Traditional RNN-based series forecasting methods struggle to effectively address long-term dependencies and gradient issues in long time series problems. Recently, SegRNN has emerged as a leading RNN-based model tailored for long-term series forecasting, demonstrating state-of-the-art performance while maintaining a streamlined architecture through innovative segmentation and parallel decoding techniques. Nevertheless, SegRNN has several limitations: its fixed segmentation disrupts data continuity and fails to effectively leverage information across different segments, the segmentation strategy employed by SegRNN does not fundamentally address the issue of information loss within the recurrent structure. To address these issues, we propose the ISMRNN method with three key enhancements: we introduce an implicit segmentation structure to decompose the time series and map it to segmented hidden states, resulting in denser information exchange during the segmentation phase. Additionally, we incorporate residual structures in the encoding layer to mitigate information loss within the recurrent structure. To extract information more effectively, we further integrate the Mamba architecture to enhance time series information extraction. Experiments on several real-world long time series forecasting datasets demonstrate that our model surpasses the performance of current state-of-the-art models.
Paper Structure (24 sections, 6 equations, 5 figures, 6 tables)

This paper contains 24 sections, 6 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Transformer Models
  4. MLP Models
  5. CNN Models
  6. RNN Models
  7. Method
  8. Overview
  9. Implicit Segmentation
  10. Residual Structure
  11. Mamba structure
  12. Experiments
  13. Datasets
  14. Experimental setup and Baselines
  15. Main Result
  16. ...and 9 more sections

Figures (5)

  • Figure 1: Overall structure of ISMRNN model.
  • Figure 2: Comparsion of SegRNN Segmentation with Implicit Segmentation, we utilize a linear layers, from 1 to $n$, to decompose the time series. Subsequently, a second mapping is applied to compress the dimensions. Note that each segmented vector embedding can encompass information from other segments of the original time series.
  • Figure 3: Comparsion of SegRNN and ISMRNN with vary look-back window length.
  • Figure 4: Comparison of spatiotemporal efficiency across different models.
  • Figure 5: Prediction result of different method. The four methods are Dlinear, ISMRNN, SegRNN, and PatchTST in sequence. Intuitively, our method also has the best fitting performance, which is consistent with the fact that ISMRNN has the minimum loss among the models.