Bi-Mamba+: Bidirectional Mamba for Time Series Forecasting

TL;DR

The paper tackles long-term multivariate time series forecasting by integrating state-space modeling with patch-based, bidirectional tokenization. It introduces Bi-Mamba+, featuring a forget-enhanced Mamba+ block, a bidirectional encoder, and a Series-Relation-Aware (SRA) decider to adapt between channel-independent and channel-mixing tokenization strategies based on dataset correlations, with patch-wise tokens to capture finer temporal dynamics. Empirical results on 8 real-world datasets show competitive or superior accuracy and improved efficiency compared to Transformer- and Mamba-based baselines, with ablations confirming the value of each component. The work advances scalable, robust LTSF by balancing intra- and inter-series dependencies and enabling hardware-aware, efficient sequence modeling for long horizons.

Abstract

Long-term time series forecasting (LTSF) provides longer insights into future trends and patterns. Over the past few years, deep learning models especially Transformers have achieved advanced performance in LTSF tasks. However, LTSF faces inherent challenges such as long-term dependencies capturing and sparse semantic characteristics. Recently, a new state space model (SSM) named Mamba is proposed. With the selective capability on input data and the hardware-aware parallel computing algorithm, Mamba has shown great potential in balancing predicting performance and computational efficiency compared to Transformers. To enhance Mamba's ability to preserve historical information in a longer range, we design a novel Mamba+ block by adding a forget gate inside Mamba to selectively combine the new features with the historical features in a complementary manner. Furthermore, we apply Mamba+ both forward and backward and propose Bi-Mamba+, aiming to promote the model's ability to capture interactions among time series elements. Additionally, multivariate time series data in different scenarios may exhibit varying emphasis on intra- or inter-series dependencies. Therefore, we propose a series-relation-aware decider that controls the utilization of channel-independent or channel-mixing tokenization strategy for specific datasets. Extensive experiments on 8 real-world datasets show that our model achieves more accurate predictions compared with state-of-the-art methods.

Figures (8)

• Figure 1: The architecture of Bi-Mamba+. The input time series is first divided into patches and then pass through a channel-independent or channel-mixing embedding layer based on the data characteristics of specific datasets. The embeddings are then fed into multiple Bi-Mamba Encoders and get the final output through an MLP projector.
• Figure 2: The architecture of (a) Bi-Mamba+ encoder and (b) Mamba+ block.
• Figure 3: The tokenization strategy indicator $ts$ of different datasets with varying settings of filter threshold $\lambda$.
• Figure 4: Results of varying patch length $P$ on Weather, Electricity, ETTh1 and ETTh2.
• Figure 5: Results of varying dropout, hidden dimension N and kernel size values on Electricity and ETTh2.