Deep Generative Imputation Model for Missing Not At Random Data

TL;DR

This work tackles MNAR data imputation by reframing the relationship between data and missingness as a multimodal problem. It introduces the conjunction model and the GNR framework, which uses parallel decoders for data and mask conditioned on a shared latent representation, enabling unbiased learning via an importance-weighted variational lower bound. Empirically, GNR achieves state-of-the-art RMSE performance across synthetic and real-world MNAR settings and delivers superior mask reconstruction, providing principled and robust imputations. The approach also offers interpretable latent-space dynamics and a practical hyperparameter guide, with potential extensions to supervised learning and causal discovery.

Abstract

Data analysis usually suffers from the Missing Not At Random (MNAR) problem, where the cause of the value missing is not fully observed. Compared to the naive Missing Completely At Random (MCAR) problem, it is more in line with the realistic scenario whereas more complex and challenging. Existing statistical methods model the MNAR mechanism by different decomposition of the joint distribution of the complete data and the missing mask. But we empirically find that directly incorporating these statistical methods into deep generative models is sub-optimal. Specifically, it would neglect the confidence of the reconstructed mask during the MNAR imputation process, which leads to insufficient information extraction and less-guaranteed imputation quality. In this paper, we revisit the MNAR problem from a novel perspective that the complete data and missing mask are two modalities of incomplete data on an equal footing. Along with this line, we put forward a generative-model-specific joint probability decomposition method, conjunction model, to represent the distributions of two modalities in parallel and extract sufficient information from both complete data and missing mask. Taking a step further, we exploit a deep generative imputation model, namely GNR, to process the real-world missing mechanism in the latent space and concurrently impute the incomplete data and reconstruct the missing mask. The experimental results show that our GNR surpasses state-of-the-art MNAR baselines with significant margins (averagely improved from 9.9% to 18.8% in RMSE) and always gives a better mask reconstruction accuracy which makes the imputation more principle.

Figures (6)

• Figure 1: Histogram of the distribution of observed and unobserved (or missing) data shows obvious differences on two five-star rating datasets (Yahoo!R3 and Coat$\text{)}^{\P}$.
• Figure 2: The procedure of inference with deep generative imputation models based on selection model or conjunction model. The selection-model-based methods use a serial structure to fit the definition of Equation \ref{['4']}. The dashed lines are some specific examples of value inference. The reconstructed mask mapping from the imputed data is biased because of the wrong distribution matching. While the conjunction-model-based GNR avoids serial mapping and adopts a parallel structure (Equation \ref{['8']}) to extract the information both in the data space and mask space without interfering with each other.
• Figure 3: Graphical representation of GNR.
• Figure 4: Visualization of latent space. (a): the value distribution of latent space after dimension reduction; (b): the smooth kernel density of the two dimensions in (a).
• Figure 5: Random local details of the reconstructed undiscretized mask in 6 realistic datasets and 1 synthetic dataset. The value in the undiscretized mask can be considered as the probability that a value is observed, namely probabilistic mask.