SiloFuse: Cross-silo Synthetic Data Generation with Latent Tabular Diffusion Models

TL;DR

SiloFuse addresses the challenge of cross-silo, vertically partitioned tabular data synthesis by adopting a distributed latent diffusion framework that operates on latent embeddings rather than raw features. Autoencoders encode client data into continuous latents, which are centralized and modeled with a diffusion backbone, while local decoders recover synthetic data at each client, preserving privacy and data on-premise. The framework employs a stacked two-step training regime that decouples autoencoder and diffusion training, reducing communication rounds to a single round and mitigating gradient leakage risks. Empirical results on nine datasets show SiloFuse achieving competitive resemblance and downstream utility compared with centralized diffusion methods, with notable gains over GAN baselines, and demonstrate robustness to client re-partitioning and privacy considerations when synthetic data is shared post-generation.

Abstract

Synthetic tabular data is crucial for sharing and augmenting data across silos, especially for enterprises with proprietary data. However, existing synthesizers are designed for centrally stored data. Hence, they struggle with real-world scenarios where features are distributed across multiple silos, necessitating on-premise data storage. We introduce SiloFuse, a novel generative framework for high-quality synthesis from cross-silo tabular data. To ensure privacy, SiloFuse utilizes a distributed latent tabular diffusion architecture. Through autoencoders, latent representations are learned for each client's features, masking their actual values. We employ stacked distributed training to improve communication efficiency, reducing the number of rounds to a single step. Under SiloFuse, we prove the impossibility of data reconstruction for vertically partitioned synthesis and quantify privacy risks through three attacks using our benchmark framework. Experimental results on nine datasets showcase SiloFuse's competence against centralized diffusion-based synthesizers. Notably, SiloFuse achieves 43.8 and 29.8 higher percentage points over GANs in resemblance and utility. Experiments on communication show stacked training's fixed cost compared to the growing costs of end-to-end training as the number of training iterations increases. Additionally, SiloFuse proves robust to feature permutations and varying numbers of clients.

Figures (11)

• Figure 1: Synthetic data for data sharing and augmentation. Features from a cardiac center (client 1) and a psychiatric center (client 2) are centralized from the common patient IDs and a generative model synthesizes new samples to share with the clients, augmenting their datasets and enabling collaboration without sharing real data.
• Figure 2: Forward and backward diffusion process. $X^0$ is an input dataset of samples and $X^T$ is the fully-noised dataset after $T$ timesteps.
• Figure 3: Design of TabDDPM. Internally, the model predicts the noise added to $x_i^t$ to minimize the difference with the original inputs after denoising.
• Figure 4: Centralized latent tabular DDPM. $Z$ denotes the latent space.
• Figure 5: Centralized latent DDPM. While $\mathcal{E}$ is the encoder, there is also a decoder $\mathcal{D}$ (not shown) for translating latents back to the original space. Internally, the DDPM predicts the noise added during the forward process, which minimizes the gap between $z_i^0$ and $\tilde{z}_i^0$

Theorems & Definitions (2)

• Example 2.1
• Example 2.2