MODE: Efficient Time Series Prediction with Mamba Enhanced by Low-Rank Neural ODEs

TL;DR

MODE addresses the challenge of accurate, efficient forecasting for irregularly sampled and long-sequence time series by unifying Low-Rank Neural ODEs with the Enhanced Mamba architecture. It introduces a continuous-time, low-rank state transition $A(t)=U(t)V(t)^ op$ with input-conditioned parameters, coupled with a segmented selective scanning mechanism to focus computation on informative subsequences. The model achieves state-of-the-art accuracy and improves scalability and robustness across diverse benchmarks, while reducing the computational burden from $O(d^2)$ to $O(d r)$ and sequence processing from $O(L d^2)$ to $O(L d log k)$. Theoretical guarantees on representation, stability, generalization, and efficiency, along with extensive empirical validations, demonstrate the practicality and impact of MODE for real-world time series prediction. Future work includes probabilistic inference and physics-informed priors to enhance uncertainty quantification and applicability.

Abstract

Time series prediction plays a pivotal role across diverse domains such as finance, healthcare, energy systems, and environmental modeling. However, existing approaches often struggle to balance efficiency, scalability, and accuracy, particularly when handling long-range dependencies and irregularly sampled data. To address these challenges, we propose MODE, a unified framework that integrates Low-Rank Neural Ordinary Differential Equations (Neural ODEs) with an Enhanced Mamba architecture. As illustrated in our framework, the input sequence is first transformed by a Linear Tokenization Layer and then processed through multiple Mamba Encoder blocks, each equipped with an Enhanced Mamba Layer that employs Causal Convolution, SiLU activation, and a Low-Rank Neural ODE enhancement to efficiently capture temporal dynamics. This low-rank formulation reduces computational overhead while maintaining expressive power. Furthermore, a segmented selective scanning mechanism, inspired by pseudo-ODE dynamics, adaptively focuses on salient subsequences to improve scalability and long-range sequence modeling. Extensive experiments on benchmark datasets demonstrate that MODE surpasses existing baselines in both predictive accuracy and computational efficiency. Overall, our contributions include: (1) a unified and efficient architecture for long-term time series modeling, (2) integration of Mamba's selective scanning with low-rank Neural ODEs for enhanced temporal representation, and (3) substantial improvements in efficiency and scalability enabled by low-rank approximation and dynamic selective scanning.

Figures (6)

• Figure 1: Low-Rank neural ODE powered Mamba
• Figure 2: The overall framework of the proposed MODE model. MODE integrates temporal embeddings, low-rank decomposition, and selective scan mechanisms within each segment to efficiently capture both local and global temporal dependencies. The pipeline consists of four main modules: (1) Embedding, which projects the input time series into latent representations; (2) U, V, B, C Generation, which generates low-rank factor matrices through depthwise and pointwise convolutions; (3) Low-Rank ODE in Segment, which performs dynamic selective scanning to model intra-segment dependencies; and (4) Gating & Concat, which aggregates outputs across segments with adaptive gating for the final prediction. This design enables MODE to achieve both efficiency and robustness in long-term time series forecasting.
• Figure 3: Efficiency comparison of our method MODE and baselines on four datasets in terms of training time.
• Figure 4: Robustness experiments of MODE on ETTm1 with varied standard deviations of gaussian noise.
• Figure 5: Long-term prediction comparison experiments of MODE on datasets with increasing lookback length.

Theorems & Definitions (5)

• proof
• proof
• proof
• proof
• proof