Functional Latent Dynamics for Irregularly Sampled Time Series Forecasting

TL;DR

This paper addresses forecasting irregularly sampled multivariate time series with missing values by introducing Functional Latent Dynamics (FLD), which represents latent dynamics with simple, continuous-time curves whose coefficients are learned from observed data via an attention-based encoder. Unlike traditional neural ODEs, FLD avoids slow solvers and sequential processing, offering faster inference while maintaining competitive accuracy across four real-world IMTS benchmarks. FLD supports multiple curve families (linear, quadratic, sine) and uses continuous time embeddings to handle irregular timing, including missing values, with an efficient decoder to generate forecasts. The approach demonstrates significant efficiency gains over ODE-based models and competitive performance against state-of-the-art IMTS methods, and points toward extensions to probabilistic forecasting and curve-selection learning for datasets.

Abstract

Irregularly sampled time series with missing values are often observed in multiple real-world applications such as healthcare, climate and astronomy. They pose a significant challenge to standard deep learning models that operate only on fully observed and regularly sampled time series. In order to capture the continuous dynamics of the irregular time series, many models rely on solving an Ordinary Differential Equation (ODE) in the hidden state. These ODE-based models tend to perform slow and require large memory due to sequential operations and a complex ODE solver. As an alternative to complex ODE-based models, we propose a family of models called Functional Latent Dynamics (FLD). Instead of solving the ODE, we use simple curves which exist at all time points to specify the continuous latent state in the model. The coefficients of these curves are learned only from the observed values in the time series ignoring the missing values. Through extensive experiments, we demonstrate that FLD achieves better performance compared to the best ODE-based model while reducing the runtime and memory overhead. Specifically, FLD requires an order of magnitude less time to infer the forecasts compared to the best performing forecasting model.

Figures (5)

• Figure 1: Example for regularly and irregularly sampled Time Series with two channels. The observations and forecasting targets are marked as black crosses.
• Figure 2: Example of FLD with sine functions as a 3-dimensional hidden state. The parameters $\theta$ of the hidden state function $g(\cdot; \theta)$ are inferred by aggregating the observations (red/blue dots) with the attention-based FLD-Encoder. The hidden state at the query times is acquired by following $g(t^q; \theta)$ and decoded by a neural network ($\text{NN}^\text{out}$).
• Figure 3: FLD-Encoder infers coefficients $\theta$ to model the hidden dynamics of an IMTS. The channel observations are aggregated with attention (Attn), concatenated ($//$) and combined with a feed forward layer (FF).
• Figure 4: Experiment on synthetic data created by the Goodwin oscillator model. We show FLD-L's forecast (left) and the inferred hidden states (right).
• Figure 5: Efficiency comparison of FLD-L and GraFITi. We plot the validation loss and inference time for 10 randomly sampled hyperparameter configurations for each GraFITi and FLD-L. The plots refer to results on the 75%-25% task on MIMIC-III and MIMIC-IV.