Investigating a Model-Agnostic and Imputation-Free Approach for Irregularly-Sampled Multivariate Time-Series Modeling

TL;DR

This work tackles irregularly-sampled multivariate time series (IMTS) by proposing MissTSM, a model-agnostic and imputation-free wrapper that harnesses a Time-Feature Independent (TFI) embedding and a Missing Feature-Aware Attention (MFAA) mechanism to handle missing values without imputations. MissTSM operates as a plug-in layer that can wrap any backbone MTS model, with a Masked Auto-Encoder (MAE) backbone used as a default. Through extensive experiments on forecasting and classification across synthetic and real-world datasets, MissTSM demonstrates competitive performance, particularly when missingness is high or periodic structure is weak, and shows strong model-agnostic versatility compared to both imputation-based and specialized IMTS approaches. The study highlights MissTSM’s potential to reduce artifacts from imputation, improve robustness to missing data patterns, and enable flexible integration with state-of-the-art MTS models for practical IMTS applications.

Abstract

Modeling Irregularly-sampled and Multivariate Time Series (IMTS) is crucial across a variety of applications where different sets of variates may be missing at different time-steps due to sensor malfunctions or high data acquisition costs. Existing approaches for IMTS either consider a two-stage impute-then-model framework or involve specialized architectures specific to a particular model and task. We perform a series of experiments to derive novel insights about the performance of IMTS methods on a variety of semi-synthetic and real-world datasets for both classification and forecasting. We also introduce Missing Feature-aware Time Series Modeling (MissTSM) or MissTSM, a novel model-agnostic and imputation-free approach for IMTS modeling. We show that MissTSM shows competitive performance compared to other IMTS approaches, especially when the amount of missing values is large and the data lacks simplistic periodic structures - conditions common to real-world IMTS applications.

Figures (24)

• Figure 1: We investigate the relative importance of three categories of approaches for modeling irregular and multivariate time-series: (1) imputation-based approaches, (2) model-agnostic and imputation-free approaches (proposed MissTSM layer), and (3) imputation-free approaches involving specialized architectures.
• Figure 2: Schematic of the Time-Feature Independent (TFI) Embedding of MissTSM that learns a different embedding for every combination of time-step and variate, in contrast to the time-only embeddings of Transformer vaswani2017attention and the variate-only embeddings of iTransformers liu2023itransformer.
• Figure 3: Overview of the MissTSM layer integrated within the Masked Auto-Encoder framework li2023ti. A zoomed-in view of the MFAA is shown on the left.
• Figure 4: Performance comparison against different TS Baselines imputed with SAITS, across different missing data fractions.
• Figure 5: Classification F1 scores on three datasets, EMG, Epilepsy, and Gesture. Masking fractions considered: 0.2, 0.4, 0.6, 0.8.