IVP-VAE: Modeling EHR Time Series with Initial Value Problem Solvers

TL;DR

IVP-VAE targets irregularly sampled EHR time series by replacing sequential RNN-based processing with parallel evolution of multiple IVPs to model a latent continuous process. A single, invertible IVP solver is shared between the encoder and decoder within a VAE, enabling parameter sharing, faster convergence, and substantial speedups while maintaining or improving forecasting and mortality classification performance across three real-world datasets. The approach yields strong data efficiency, particularly in small-data regimes, and demonstrates robustness across domains by outperforming latent-based baselines and remaining competitive with state-of-the-art methods. Overall, IVP-VAE offers a scalable, efficient framework for continuous-time representation learning on irregular time series with practical clinical impact.

Abstract

Continuous-time models such as Neural ODEs and Neural Flows have shown promising results in analyzing irregularly sampled time series frequently encountered in electronic health records. Based on these models, time series are typically processed with a hybrid of an initial value problem (IVP) solver and a recurrent neural network within the variational autoencoder architecture. Sequentially solving IVPs makes such models computationally less efficient. In this paper, we propose to model time series purely with continuous processes whose state evolution can be approximated directly by IVPs. This eliminates the need for recurrent computation and enables multiple states to evolve in parallel. We further fuse the encoder and decoder with one IVP solver utilizing its invertibility, which leads to fewer parameters and faster convergence. Experiments on three real-world datasets show that the proposed method can systematically outperform its predecessors, achieve state-of-the-art results, and have significant advantages in terms of data efficiency.

Figures (4)

• Figure 1: Modeling irregular time series with IVP-VAE. (Left) In the encoder, an embedding module maps data $\boldsymbol{x}_i$ into latent state $\boldsymbol{z}_i$. The state is evolved backward in time: Take $(\boldsymbol{z}_i, t_i)$ as initial condition and calculate state $\boldsymbol{z}_0$ at $t_0$ using an IVP solver. (Right) In the decoder, the latent state is evolved forward in time: Take $(\boldsymbol{z}_0, t_0)$ as initial condition and go opposite along the timeline to obtain state $\boldsymbol{z}_i$ using the same IVP solver. A reconstruction module then maps $\boldsymbol{z}_i$ back to data $\boldsymbol{\hat{x}}_i$.
• Figure 2: Performance comparison on small datasets: (Left) MSE for forecasting and (right) AUROC for classification task. IVP-VAE based models consistently and substantially outperform all baseline approaches across all datasets with different number of samples.
• Figure 3: Evolving forward and backward in time using the same IVP solver and different initial points
• Figure 4: Samples of the synthetic dataset