TabDDPM: Modelling Tabular Data with Diffusion Models

TL;DR

TabDDPM extends denoising diffusion probabilistic models to tabular data, enabling a unified framework that handles mixed numerical and categorical features via Gaussian diffusion for numbers and multinomial diffusion for categories.The model demonstrates state-of-the-art synthetic-data quality on diverse tabular benchmarks, outperforming GAN- and VAE-based competitors while presenting a privacy-friendly alternative to real data sharing.A thorough evaluation using ML efficiency and privacy metrics shows TabDDPM often yields superior downstream model performance, though simple SMOTE remains a competitive baseline in some scenarios.The work also discusses privacy-utility trade-offs and limitations, suggesting future directions in diffusion variants and more refined privacy assessments for tabular data synthesis.

Abstract

Denoising diffusion probabilistic models are currently becoming the leading paradigm of generative modeling for many important data modalities. Being the most prevalent in the computer vision community, diffusion models have also recently gained some attention in other domains, including speech, NLP, and graph-like data. In this work, we investigate if the framework of diffusion models can be advantageous for general tabular problems, where datapoints are typically represented by vectors of heterogeneous features. The inherent heterogeneity of tabular data makes it quite challenging for accurate modeling, since the individual features can be of completely different nature, i.e., some of them can be continuous and some of them can be discrete. To address such data types, we introduce TabDDPM -- a diffusion model that can be universally applied to any tabular dataset and handles any type of feature. We extensively evaluate TabDDPM on a wide set of benchmarks and demonstrate its superiority over existing GAN/VAE alternatives, which is consistent with the advantage of diffusion models in other fields. Additionally, we show that TabDDPM is eligible for privacy-oriented setups, where the original datapoints cannot be publicly shared.

Figures (6)

• Figure 1: TabDDPM scheme for classification problems; $t$, $y$ and $\ell$ denote a diffusion timestep, a class label, and logits, respectively.
• Figure 2: The individual feature distributions for the real data and the data generated by TabDDPM, CTABGAN+, and TVAE. TabDDPM produces more realistic feature distributions than alternatives in most cases.
• Figure 3: Absolute difference between correlation matrices computed on real and synthetic datasets. A more intensive red color indicates a higher difference between the real and synthetic correlation values. In most cases, TabDDPM captures feature correlations better.
• Figure 4: Histograms of minimal synthetic-to-real distances for TabDDPM and SMOTE. SMOTE values are concentrated around zero and, thus, SMOTE generates less private synthetic data.
• Figure 5: The individual feature distributions for the real data and the data generated by TabDDPM, CTABGAN+, and TVAE. TabDDPM often models feature distributions more accurately than CTABGAN+ and TVAE.