UmambaTSF: A U-shaped Multi-Scale Long-Term Time Series Forecasting Method Using Mamba

TL;DR

UmambaTSF is proposed, a novel long-term time series forecasting framework that integrates multi-scale feature extraction capabilities of U-shaped encoder-decoder multilayer perceptrons (MLP) with Mamba's long sequence representation, and achieves state-of-the-art performance and excellent generality on widely used benchmark datasets while maintaining linear time complexity and low memory consumption.

Abstract

Multivariate Time series forecasting is crucial in domains such as transportation, meteorology, and finance, especially for predicting extreme weather events. State-of-the-art methods predominantly rely on Transformer architectures, which utilize attention mechanisms to capture temporal dependencies. However, these methods are hindered by quadratic time complexity, limiting the model's scalability with respect to input sequence length. This significantly restricts their practicality in the real world. Mamba, based on state space models (SSM), provides a solution with linear time complexity, increasing the potential for efficient forecasting of sequential data. In this study, we propose UmambaTSF, a novel long-term time series forecasting framework that integrates multi-scale feature extraction capabilities of U-shaped encoder-decoder multilayer perceptrons (MLP) with Mamba's long sequence representation. To improve performance and efficiency, the Mamba blocks introduced in the framework adopt a refined residual structure and adaptable design, enabling the capture of unique temporal signals and flexible channel processing. In the experiments, UmambaTSF achieves state-of-the-art performance and excellent generality on widely used benchmark datasets while maintaining linear time complexity and low memory consumption.

Figures (8)

• Figure 1: Average performance (MSE) between UmambaTSF and the latest SOTA models on seven public real-world datasets. The center of the circle denotes the maximum error, while points nearer to the boundary indicate better performance.
• Figure 2: Overall framework of UmambaTSF, where the top-left section illustrates the overall architecture of our model. The center-left and bottom-left sections introduce the multi-scale feature extractor and the Mamba-based temporal signal processor (MTSP), which are used to extract temporal features at each scale. The top-right section provides an explanation of the operation symbols, while the bottom-right section describes the structure of the Mamba model.
• Figure 3: Schematic Diagrams of Channel Processing in Three Different Scenarios.
• Figure 4: Comparison of forecasts among UmambaTSF, iTransformer, and S-Mamba on four datasets, using an input length of 96 and a forecast length of 96. The blue line depicts the ground truth, while the red, black, and purple lines represent the predictions from UmambaTSF, S-Mamba, and iTransformer, respectively.
• Figure 5: Comparison of UmambaTSF and five models on MSE, Computational Complexity, and GPU Memory. The diameters of the circles represent the memory sizes.