ODEStream: A Buffer-Free Online Learning Framework with ODE-based Adaptor for Streaming Time Series Forecasting

TL;DR

ODEStream addresses irregularity and concept drift in streaming time series by offering a buffer-free online continual learning framework that combines Neural ODEs with a temporal isolation layer to model continuous dynamics without retaining past samples. The method uses a two-phase process: an offline warm-up with a variational autoencoder–ODE to capture prior dynamics and an online phase that continuously adapts to new data with no external memory, guided by a composite loss including $\mathcal{L}_{MSE}$, $\mathcal{L}_{KL}$, and $\mathcal{L}_{L1}$. Empirical results on real-world datasets show that ODEStream outperforms state-of-the-art online baselines in long-horizon forecasting and rapid adaptation to drift, while remaining robust to irregular sampling and communication latencies, and it does so with favorable computational efficiency. The work advances real-time streaming analysis by leveraging continuous-time modelling for time-varying distributions, and it provides an open-source implementation for broader adoption. Future work includes extending the framework to task-incremental learning scenarios with multiple labeled time-series tasks.

Abstract

Addressing the challenges of irregularity and concept drift in streaming time series is crucial for real-world predictive modelling. Previous studies in time series continual learning often propose models that require buffering long sequences, potentially restricting the responsiveness of the inference system. Moreover, these models are typically designed for regularly sampled data, an unrealistic assumption in real-world scenarios. This paper introduces ODEStream, a novel buffer-free continual learning framework that incorporates a temporal isolation layer to capture temporal dependencies within the data. Simultaneously, it leverages the capability of neural ordinary differential equations to process irregular sequences and generate a continuous data representation, enabling seamless adaptation to changing dynamics in a data streaming scenario. Our approach focuses on learning how the dynamics and distribution of historical data change over time, facilitating direct processing of streaming sequences. Evaluations on benchmark real-world datasets demonstrate that ODEStream outperforms the state-of-the-art online learning and streaming analysis baseline models, providing accurate predictions over extended periods while minimising performance degradation over time by learning how the sequence dynamics change. The implementation of ODEStream is available at: https://github.com/FtoonAbushaqra/ODEStream.git.

Figures (8)

• Figure 1: ODEStream Framework: which leverages Neural ODEs to encode prior knowledge during the initial training phase. Subsequently, the learned model parameters are transferred and complemented by the temporal isolation layer for continuous online learning. In real-time, streaming data samples are used to predict future values, while the model simultaneously updates its parameters to adapt to newly observed data.
• Figure 2: The continual learning process of ODEStream. The model processes the incoming observation using the pre-trained VAE-ODE and continuously changes its parameters based on new dynamics.
• Figure 3: ODEStream against FSNet: Adapting dynamics evolution for the entire stream sequence of several datasets.
• Figure 4: ODEStream against DER++: Adapting dynamics evolution for the entire stream sequence of several datasets.
• Figure 5: Model performance on irregularly sampled data.