Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Observations: Reconstruction Error-Guided Irregularly Sampled Time Series Representation Learning

Jiexi Liu, Meng Cao, Songcan Chen

TL;DR

This work tackles irregularly sampled time series (ISTS) by exploiting reconstruction error as a self-supervised learning signal. It introduces iTimER, a pretraining framework that models the distribution of reconstruction errors on observed timestamps and generates uncertainty-aware pseudo-observations for missing timestamps through a mixup strategy, coupled with distribution alignment and contrastive learning. The approach yields a robust encoder for ISTS that transfers to classification, interpolation, and forecasting tasks, outperforming state-of-the-art baselines across multiple datasets while maintaining efficiency. By leveraging reconstruction error as guidance, iTimER provides a principled way to learn from imperfect, nonuniformly sampled data and offers potential as a plug-and-play module to enhance ISTS models under distribution shift and noisy conditions.

Abstract

Irregularly sampled time series (ISTS), characterized by non-uniform time intervals with natural missingness, are prevalent in real-world applications. Existing approaches for ISTS modeling primarily rely on observed values to impute unobserved ones or infer latent dynamics. However, these methods overlook a critical source of learning signal: the reconstruction error inherently produced during model training. Such error implicitly reflects how well a model captures the underlying data structure and can serve as an informative proxy for unobserved values. To exploit this insight, we propose iTimER, a simple yet effective self-supervised pre-training framework for ISTS representation learning. iTimER models the distribution of reconstruction errors over observed values and generates pseudo-observations for unobserved timestamps through a mixup strategy between sampled errors and the last available observations. This transforms unobserved timestamps into noise-aware training targets, enabling meaningful reconstruction signals. A Wasserstein metric aligns reconstruction error distributions between observed and pseudo-observed regions, while a contrastive learning objective enhances the discriminability of learned representations. Extensive experiments on classification, interpolation, and forecasting tasks demonstrate that iTimER consistently outperforms state-of-the-art methods under the ISTS setting.

Beyond Observations: Reconstruction Error-Guided Irregularly Sampled Time Series Representation Learning

TL;DR

This work tackles irregularly sampled time series (ISTS) by exploiting reconstruction error as a self-supervised learning signal. It introduces iTimER, a pretraining framework that models the distribution of reconstruction errors on observed timestamps and generates uncertainty-aware pseudo-observations for missing timestamps through a mixup strategy, coupled with distribution alignment and contrastive learning. The approach yields a robust encoder for ISTS that transfers to classification, interpolation, and forecasting tasks, outperforming state-of-the-art baselines across multiple datasets while maintaining efficiency. By leveraging reconstruction error as guidance, iTimER provides a principled way to learn from imperfect, nonuniformly sampled data and offers potential as a plug-and-play module to enhance ISTS models under distribution shift and noisy conditions.

Abstract

Irregularly sampled time series (ISTS), characterized by non-uniform time intervals with natural missingness, are prevalent in real-world applications. Existing approaches for ISTS modeling primarily rely on observed values to impute unobserved ones or infer latent dynamics. However, these methods overlook a critical source of learning signal: the reconstruction error inherently produced during model training. Such error implicitly reflects how well a model captures the underlying data structure and can serve as an informative proxy for unobserved values. To exploit this insight, we propose iTimER, a simple yet effective self-supervised pre-training framework for ISTS representation learning. iTimER models the distribution of reconstruction errors over observed values and generates pseudo-observations for unobserved timestamps through a mixup strategy between sampled errors and the last available observations. This transforms unobserved timestamps into noise-aware training targets, enabling meaningful reconstruction signals. A Wasserstein metric aligns reconstruction error distributions between observed and pseudo-observed regions, while a contrastive learning objective enhances the discriminability of learned representations. Extensive experiments on classification, interpolation, and forecasting tasks demonstrate that iTimER consistently outperforms state-of-the-art methods under the ISTS setting.

Paper Structure

This paper contains 21 sections, 8 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Representation Learning for Irregularly Sampled Time Series
  4. Reconstruction Error as a Learning Signal
  5. Proposed iTimER Framework
  6. Problem Setup and Method Outline
  7. Pseudo-Observation Learning
  8. Modeling Reconstruction Uncertainty.
  9. Error Sampling at Unobserved Timestamps.
  10. Constructing Complete Pseudo-Observation Series.
  11. Learning Objective Design
  12. Experiments
  13. Experimental Setup
  14. Datasets and Metrics.
  15. Baselines.
  16. ...and 6 more sections

Figures (3)

  • Figure 1: Multivariate irregularly sampled time series (ISTS) with two variables $C_1$ and $C_2$, exhibiting non-uniform sampling intervals with natural missingness.
  • Figure 2: The iTimER framework. iTimER leverages reconstruction error from the original sequence to generate pseudo observations via mixup uncertainty-aware sampling and the last observed value. Both real and pseudo-observation series are encoded and reconstructed, with consistency enforced at both representation and error distribution levels.
  • Figure 3: Efficiency comparisons in terms of Training Time (s) and Memory Usage (G) with the latest advanced models on the P12 datasets.