Debiasing Synthetic Data Generated by Deep Generative Models
Alexander Decruyenaere, Heidelinde Dehaene, Paloma Rabaey, Christiaan Polet, Johan Decruyenaere, Thomas Demeester, Stijn Vansteelandt
TL;DR
This paper tackles the challenge that deep generative models introduce bias and slow convergence when deriving statistical inferences from synthetic tabular data. It proposes a generator-agnostic debiasing framework rooted in efficient influence curves and targeted learning, combining a post-processing data shift with EIC-based estimators to achieve more honest inference from a single synthetic dataset. The authors establish theoretical von Mises expansions, provide a practical variance expression, and demonstrate via simulations and real-data case studies that debiasing improves coverage for population means and often for regression coefficients, with SEs that converge near the root-$n$ rate under debiasing. The approach remains practical, generator-agnostic, and points to extensions such as sample-splitting, importance weighting, and DP considerations to broaden applicability to privacy-preserving settings.
Abstract
While synthetic data hold great promise for privacy protection, their statistical analysis poses significant challenges that necessitate innovative solutions. The use of deep generative models (DGMs) for synthetic data generation is known to induce considerable bias and imprecision into synthetic data analyses, compromising their inferential utility as opposed to original data analyses. This bias and uncertainty can be substantial enough to impede statistical convergence rates, even in seemingly straightforward analyses like mean calculation. The standard errors of such estimators then exhibit slower shrinkage with sample size than the typical 1 over root-$n$ rate. This complicates fundamental calculations like p-values and confidence intervals, with no straightforward remedy currently available. In response to these challenges, we propose a new strategy that targets synthetic data created by DGMs for specific data analyses. Drawing insights from debiased and targeted machine learning, our approach accounts for biases, enhances convergence rates, and facilitates the calculation of estimators with easily approximated large sample variances. We exemplify our proposal through a simulation study on toy data and two case studies on real-world data, highlighting the importance of tailoring DGMs for targeted data analysis. This debiasing strategy contributes to advancing the reliability and applicability of synthetic data in statistical inference.