Temporal Dynamic Embedding for Irregularly Sampled Time Series

TL;DR

This work addresses the challenge of irregularly sampled multivariate time series by introducing Temporal Dynamic Embedding (TDE), which represents each variable as a time-evolving latent embedding and aggregates only observed observations at each time step. Through variable embeddings and time encodings, TDE forms a dynamic status S^t that feeds a GRU-like temporal state for online classification, avoiding imputation. The method advances two aggregation schemes—mean and attention-based—along with a robust learning framework and a loss objective that optimizes predictive likelihood. Empirical results on PhysioNet 2012, MIMIC-III, and PhysioNet 2019 show competitive or superior performance with notable training speedups, and embedding visualizations illustrate how local and global temporal information contribute to prognosis. The approach offers practical impact for real-time clinical decision support and sets the stage for future self-supervised exploration of variable relationships in time series.

Abstract

In several practical applications, particularly healthcare, clinical data of each patient is individually recorded in a database at irregular intervals as required. This causes a sparse and irregularly sampled time series, which makes it difficult to handle as a structured representation of the prerequisites of neural network models. We therefore propose temporal dynamic embedding (TDE), which enables neural network models to receive data that change the number of variables over time. TDE regards each time series variable as an embedding vector evolving over time, instead of a conventional fixed structured representation, which causes a critical missing problem. For each time step, TDE allows for the selective adoption and aggregation of only observed variable subsets and represents the current status of patient based on current observations. The experiment was conducted on three clinical datasets: PhysioNet 2012, MIMIC-III, and PhysioNet 2019. The TDE model performed competitively or better than the imputation-based baseline and several recent state-of-the-art methods with reduced training runtime.

Figures (3)

• Figure 1: Overview of temporal dynamic embedding (a) Example of irregularly sampled multivariate time series, which consisted of 3 variables (i.e., $d_m\in\{\mathrm{d}_1,\mathrm{d}_2,\mathrm{d}_3\}$). (b) Irregularly sampled time series is re-represented by aggregating observations $s_m$, which is measured at same time point $\mathrm{t}$. This concept can represent comprehensive status of $S^t$ changing over time $t$. (c) Each variable $d_m$ is embedded into latent space considering measured value $x_m$, which is changing over time. Then, $S^t$ is represented by aggregating observations' embedding at each time point.
• Figure 2: Overall architecture Each time status is aggregated by each temporal dynamic variable embedding, which leverages update for time-recurrent hidden state. Based on the hidden state of the last observed time step, the probability of a label is calculated.
• Figure 3: Two-dimensional embedding representation using t-SNE for The PhysioNet Challenge 2019. (a) Representation of the aggregation between the observed variable embedding, measurements and time embedding at each time. (b) Representation of the recurrent hidden state of the last time step.