Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Models

TL;DR

Mamba4Cast's key innovation lies in its ability to achieve strong zero-shot performance on real-world datasets while having much lower inference times than time series foundation models based on the transformer architecture.

Abstract

This paper introduces Mamba4Cast, a zero-shot foundation model for time series forecasting. Based on the Mamba architecture and inspired by Prior-data Fitted Networks (PFNs), Mamba4Cast generalizes robustly across diverse time series tasks without the need for dataset specific fine-tuning. Mamba4Cast's key innovation lies in its ability to achieve strong zero-shot performance on real-world datasets while having much lower inference times than time series foundation models based on the transformer architecture. Trained solely on synthetic data, the model generates forecasts for entire horizons in a single pass, outpacing traditional auto-regressive approaches. Our experiments show that Mamba4Cast performs competitively against other state-of-the-art foundation models in various data sets while scaling significantly better with the prediction length. The source code can be accessed at https://github.com/automl/Mamba4Cast.

Figures (5)

• Figure 1: Schematic overview of the Mamba4Cast architecture.
• Figure 2: Performance and efficiency comparison of Mamba4Cast against baseline models. (left) Distribution of MASE across 16 evaluation datasets (excluding Covid Deaths) for Mamba4Cast and five baseline models (ForecastPFN was much worse and is on a separate scale). (right) Inference time of Mamba4Cast versus Chronos-Small on synthetically generated time series (2048 series, 512 context length) for increasing prediction lengths and varying batch sizes. The results demonstrate Mamba4Cast's superior efficiency, particularly for longer prediction horizons and larger batch sizes.
• Figure 3: Critical difference diagram comparing mean MASE ranks of Mamba4Cast and baseline models across 17 time series datasets. ForecastPFN was much worse and is excluded for the sake of visibility.
• Figure 4: Qualitative analysis of Mamba4Cast's performance. (left) Demonstrates how prediction accuracy improves with increasing context length for multiplicative sine waves. (right) Illustrates the model's forecasting capabilities on two real-world time series datasets.
• Figure 5: Qualitative analysis of real-world datasets evaluated by Mamba4Cast. Blue denotes the ground-truth, red the prediction.