Revisiting Multivariate Time Series Forecasting with Missing Values

TL;DR

It is revealed that imputation without direct supervision can corrupt the underlying data distribution and actively degrade prediction accuracy, and a paradigm shift is proposed that moves away from imputation and directly predicts from the partially observed time series, Consistency-Regularized Information Bottleneck (CRIB).

Abstract

Missing values are common in real-world time series, and multivariate time series forecasting with missing values (MTSF-M) has become a crucial area of research for ensuring reliable predictions. To address the challenge of missing data, current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data. However, this framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy. In this paper, we conduct a systematic empirical study and reveal that imputation without direct supervision can corrupt the underlying data distribution and actively degrade prediction accuracy. To address this, we propose a paradigm shift that moves away from imputation and directly predicts from the partially observed time series. We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle. CRIB combines a unified-variate attention mechanism with a consistency regularization scheme to learn robust representations that filter out noise introduced by missing values while preserving essential predictive signals. Comprehensive experiments on four real-world datasets demonstrate the effectiveness of CRIB, which predicts accurately even under high missing rates. Our code is available in https://github.com/Muyiiiii/CRIB.

Figures (14)

• Figure 1: Analysis of the imputation-then-prediction paradigm on PEMS-BAY (40% missing rate). (a) t-SNE visualizations show that current imputation modules cannot recover the original data distribution and their forecasts mismatch with the prediction target, while our direct-prediction method aligns better with the target. (b, c) Correlation maps reveal that imputation fails to recover true variate correlations, whereas our method preserves underlying correlations more effectively.
• Figure 2: Overall framework of CRIB. (a) Data Augmentation creates a more challenging view of the partially observed data $X^{\text{o}}$ by generating an augmented version $X^{\text{Aug}}$. (b) The Patching Embedding layer converts the $X^{\text{o}}$ and $X^{\text{Aug}}$ into robust patch-level feature representations $H$ and $H^{\text{Aug}}$. (c) The Unified-Variate Attention mechanism models the global correlations between all the patches within $H$ and $H^{\text{Aug}}$ to produce refined representations $Z$ and $Z^{\text{Aug}}$. (d) Consistency Regularization aligns the representations from the original $Z$ and the augmented views $Z^{\text{Aug}}$. The entire process is guided by the IB principles of compactness and informativeness to produce the final forecast $\widehat{Y}$.
• Figure 3: Average MAE on PEMS-BAY and ETTh1 with point, block, and column missing patterns.
• Figure 4: Ablation and Sensitivity experiment results on PEMS-BAY dataset of CRIB.
• Figure 5: Visualization of the input and forecasting results of CRIB on the PEMS-BAY dataset with missing rates from 20% to 70%.