Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analyzing and Improving Diffusion Models for Time-Series Data Imputation: A Proximal Recursion Perspective

Zhichao Chen, Hao Wang, Fangyikang Wang, Licheng Pan, Zhengnan Li, Yunfei Teng, Haoxuan Li, Zhouchen Lin

TL;DR

A novel framework called SPIRIT (Semi-Proximal Transport Regularized time-series Imputation) is proposed, which introduces entropy-induced Bregman divergence to relax the mass preserving constraint in the Wasserstein distance, and theoretically proves the robustness of SPT against non-stationarity.

Abstract

Diffusion models (DMs) have shown promise for Time-Series Data Imputation (TSDI); however, their performance remains inconsistent in complex scenarios. We attribute this to two primary obstacles: (1) non-stationary temporal dynamics, which can bias the inference trajectory and lead to outlier-sensitive imputations; and (2) objective inconsistency, since imputation favors accurate pointwise recovery whereas DMs are inherently trained to generate diverse samples. To better understand these issues, we analyze DM-based TSDI process through a proximal-operator perspective and uncover that an implicit Wasserstein distance regularization inherent in the process hinders the model's ability to counteract non-stationarity and dissipative regularizer, thereby amplifying diversity at the expense of fidelity. Building on this insight, we propose a novel framework called SPIRIT (Semi-Proximal Transport Regularized time-series Imputation). Specifically, we introduce entropy-induced Bregman divergence to relax the mass preserving constraint in the Wasserstein distance, formulate the semi-proximal transport (SPT) discrepancy, and theoretically prove the robustness of SPT against non-stationarity. Subsequently, we remove the dissipative structure and derive the complete SPIRIT workflow, with SPT serving as the proximal operator. Extensive experiments demonstrate the effectiveness of the proposed SPIRIT approach.

Analyzing and Improving Diffusion Models for Time-Series Data Imputation: A Proximal Recursion Perspective

TL;DR

A novel framework called SPIRIT (Semi-Proximal Transport Regularized time-series Imputation) is proposed, which introduces entropy-induced Bregman divergence to relax the mass preserving constraint in the Wasserstein distance, and theoretically proves the robustness of SPT against non-stationarity.

Abstract

Diffusion models (DMs) have shown promise for Time-Series Data Imputation (TSDI); however, their performance remains inconsistent in complex scenarios. We attribute this to two primary obstacles: (1) non-stationary temporal dynamics, which can bias the inference trajectory and lead to outlier-sensitive imputations; and (2) objective inconsistency, since imputation favors accurate pointwise recovery whereas DMs are inherently trained to generate diverse samples. To better understand these issues, we analyze DM-based TSDI process through a proximal-operator perspective and uncover that an implicit Wasserstein distance regularization inherent in the process hinders the model's ability to counteract non-stationarity and dissipative regularizer, thereby amplifying diversity at the expense of fidelity. Building on this insight, we propose a novel framework called SPIRIT (Semi-Proximal Transport Regularized time-series Imputation). Specifically, we introduce entropy-induced Bregman divergence to relax the mass preserving constraint in the Wasserstein distance, formulate the semi-proximal transport (SPT) discrepancy, and theoretically prove the robustness of SPT against non-stationarity. Subsequently, we remove the dissipative structure and derive the complete SPIRIT workflow, with SPT serving as the proximal operator. Extensive experiments demonstrate the effectiveness of the proposed SPIRIT approach.
Paper Structure (51 sections, 10 theorems, 107 equations, 5 figures, 3 tables, 1 algorithm)

This paper contains 51 sections, 10 theorems, 107 equations, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Formulation
  4. Proximal Operator and Proximal Recursion
  5. Wasserstein Distance
  6. Dissipative Structure in Stochastic Dynamics
  7. Methodology
  8. Motivation Analysis
  9. Wasserstein Distance Meets Non-Stationary.
  10. Dissipative Structures Meets Optimization.
  11. Analysis DMs In the Context of TSDI
  12. Semi-Proximal Transport Framework
  13. Diversity-Eliminated Functional for Imputation
  14. Overall Workflow
  15. Main Experimental Results
  16. ...and 36 more sections

Key Result

Proposition 3.1

The imputation process for DMs can be formulated as iteratively solving the following optimization problem in a proximal operator form: where we abbreviate the candidate distribution (distribution for current iteration) $q'({\mathbf{x}}^{\text{imp}})$ and the base distribution (distribution for previous iteration) $q({\mathbf{x}}^{\text{imp}})$ as $q'$ and $q$, respectively. The term $\frac{1}{\e

Figures (5)

  • Figure 1: Transport plan comparison between OT and SPT.
  • Figure 2: Imputation results comparison vary dissipative structures.
  • Figure 3: Sensitivity analysis results on step size: $\eta$, iteration time: $\mathcal{E}$, hidden dimension of score network: $\mathrm{H}_{s_\theta}$, and patch length: $T$. The scatters and shaded areas indicate the mean and one standard deviation from the mean, respectively.
  • Figure 4: Transport plan comparison between OT and SPT. The lines and shaded areas indicate the mean and one standard deviation from the mean, respectively.
  • Figure 5: The computational time for "Score Learning" stage and "Recursive Imputation" stage. The scatters and shaded areas indicate the mean and one standard deviation from the mean, respectively.

Theorems & Definitions (15)

  • Proposition 3.1
  • Lemma 3.2
  • Proposition 3.3
  • Proposition 3.4
  • Proposition 3.5
  • Proposition : \ref{['thm:mmsProblemSolving']}
  • proof
  • Lemma : \ref{['prop:robustness']}
  • proof
  • Proposition : \ref{['prop:updatePropositionResults']}
  • ...and 5 more