Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Continuous-time Autoencoders for Regular and Irregular Time Series Imputation

Hyowon Wi, Yehjin Shin, Noseong Park

TL;DR

This work redesigns time series (variational) autoencoders based on continuous-time recurrent neural networks (RNNs) and designs an imputation method based on neural controlled differential equations (NCDEs), which shows the best imputation performance in almost all cases.

Abstract

Time series imputation is one of the most fundamental tasks for time series. Real-world time series datasets are frequently incomplete (or irregular with missing observations), in which case imputation is strongly required. Many different time series imputation methods have been proposed. Recent self-attention-based methods show the state-of-the-art imputation performance. However, it has been overlooked for a long time to design an imputation method based on continuous-time recurrent neural networks (RNNs), i.e., neural controlled differential equations (NCDEs). To this end, we redesign time series (variational) autoencoders based on NCDEs. Our method, called continuous-time autoencoder (CTA), encodes an input time series sample into a continuous hidden path (rather than a hidden vector) and decodes it to reconstruct and impute the input. In our experiments with 4 datasets and 19 baselines, our method shows the best imputation performance in almost all cases.

Continuous-time Autoencoders for Regular and Irregular Time Series Imputation

TL;DR

This work redesigns time series (variational) autoencoders based on continuous-time recurrent neural networks (RNNs) and designs an imputation method based on neural controlled differential equations (NCDEs), which shows the best imputation performance in almost all cases.

Abstract

Time series imputation is one of the most fundamental tasks for time series. Real-world time series datasets are frequently incomplete (or irregular with missing observations), in which case imputation is strongly required. Many different time series imputation methods have been proposed. Recent self-attention-based methods show the state-of-the-art imputation performance. However, it has been overlooked for a long time to design an imputation method based on continuous-time recurrent neural networks (RNNs), i.e., neural controlled differential equations (NCDEs). To this end, we redesign time series (variational) autoencoders based on NCDEs. Our method, called continuous-time autoencoder (CTA), encodes an input time series sample into a continuous hidden path (rather than a hidden vector) and decodes it to reconstruct and impute the input. In our experiments with 4 datasets and 19 baselines, our method shows the best imputation performance in almost all cases.
Paper Structure (39 sections, 15 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 39 sections, 15 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Our approach:
  3. An infinite number of autoencoders:
  4. Hidden vector vs. hidden path:
  5. Dual layer and training with missing values:
  6. Preliminaries & Related Work
  7. Neural Controlled Differential Equations
  8. Time Series Imputation
  9. RNN-based
  10. VAE-based
  11. GAN-based
  12. Self-attention-based
  13. Problem Definition
  14. Proposed Method
  15. Encoder
  16. ...and 24 more sections

Figures (4)

  • Figure 1: Overall workflow of Latent ODE and our method, CTA, to impute incomplete observations in red.
  • Figure 2: We highlight the encoder part from Fig. \ref{['fig:CTA']}.
  • Figure 3: Overall workflow of NCDEs. In (a), after creating the continuous path $X(t)$, every computation works in the continuous time domain. When calculating $\boldsymbol{h}(t_1)$ in (b), observations around $t_1$ are considered via the interpolated continuous path $X(t)$ --- some sparse transformers also consider a small number of neighboring observations in a discrete time domain 10.1145/3530811.
  • Figure 4: Overall workflow of our proposed dual-layer imputation method. We intentionally remove some elements in $\ddot{\mathbf{X}}$ for the training effect in challenging environments.