Tripletformer for Probabilistic Interpolation of Irregularly sampled Time Series

TL;DR

This work presents a novel encoder-decoder architecture called “Tripletformer” for probabilistic interpolation of irregularly sampled time series with missing values, where each element is composed of a triple of time, channel, and value.

Abstract

Irregularly sampled time series data with missing values is observed in many fields like healthcare, astronomy, and climate science. Interpolation of these types of time series is crucial for tasks such as root cause analysis and medical diagnosis, as well as for smoothing out irregular or noisy data. To address this challenge, we present a novel encoder-decoder architecture called "Tripletformer" for probabilistic interpolation of irregularly sampled time series with missing values. This attention-based model operates on sets of observations, where each element is composed of a triple of time, channel, and value. The encoder and decoder of the Tripletformer are designed with attention layers and fully connected layers, enabling the model to effectively process the presented set elements. We evaluate the Tripletformer against a range of baselines on multiple real-world and synthetic datasets and show that it produces more accurate and certain interpolations. Results indicate an improvement in negative loglikelihood error by up to 32% on real-world datasets and 85% on synthetic datasets when using the Tripletformer compared to the next best model.

Figures (5)

• Figure 1: Interpolation in Multivariate Time Series (a) and Irregularly sampled Time Series (b). In (a) all the channels in the time series are observed at times $t_1, t_2, t_5, t_6$ and we need to interpolate the values for all the channels at $t_3, t_4$. Where as in (b), channel 1 is observed at times $t_1, t_6, t_8$, and channel 3 is observed at $t_2, t_7$. We did not make any observation in channel 2. However, we need to interpolate the values for channels 1,2 and 3 at time points $t_3$, $t_4$, and $t_5$ respectively.
• Figure 2: Demonstration of IMTS as set of observations.
• Figure 3: Architectures of Multihead Attention Block (a) and Induced Multihead Attention Block (b) LL19
• Figure 4: Tripletformer architecture. Encoder (left) takes the set of observations $X$, and output their embeddings $Z^{(e)}$. Decoder (right) takes $Z^{(e)}$, target queries ($W$), and produces the mean $\mathrm{M}$ and standard deviation $\Sigma$ corresponding to $W$.
• Figure 5: Comparison of qualitative performance between Tripletformer and HETVAE. Plots are the predictions ($95^{th}$ quantile) of both the models for heart rate in all the three datasets conditioned on the $50\%$ of the time points.