CTSyn: A Foundation Model for Cross Tabular Data Generation

TL;DR

CTSyn tackles cross-table tabular data generation by integrating a cross-tabular autoencoder with a conditional latent diffusion backbone, enabling a unified latent space for heterogeneous tables. The cross-tabular autoencoder encodes per-row, multi-level metadata into $\mathbf{z} \in \mathbb{R}^{\ell \times M_{\text{agg}}}$ via a Perceiver Resampler, while a schema-conditioned diffusion model learns to generate latent samples from $[e_m, E_c]$ guidance. Pre-training on a web-scale OpenTab corpus (~5.01M rows) and fine-tuning on downstream tasks produce strong fidelity, diversity, and downstream utility, particularly in low-data regimes, demonstrating the viability of large-scale tabular foundation models. Overall, CTSyn establishes a principled framework for cross-table generalization in tabular data synthesis and highlights the potential of latent diffusion conditioned on table schemas for scalable tabular GFM development.

Abstract

Generative Foundation Models (GFMs) have achieved remarkable success in producing high-quality synthetic data for images and text. However, their application to tabular data presents significant challenges due to the heterogeneous nature of table features. Current cross-table learning frameworks struggle because they lack a generative model backbone and an effective mechanism to decode heterogeneous feature values. To address these challenges, we propose the Cross-Table Synthesizer (CTSyn), a diffusion-based generative foundation model for tabular data generation. CTSyn comprises two key components. The first is an autoencoder network that consolidates diverse tables into a unified latent space. It dynamically reconstructs table values using a table schema embedding, allowing adaptation to heterogeneous datasets. The second is a conditional latent diffusion model that generates samples from the learned latent space, conditioned on the table schema. Through large-scale pre-training, CTSyn outperforms existing table synthesizers on standard benchmarks in both utility and diversity. These results position CTSyn as a promising framework for synthetic table generation and lay the groundwork for developing large-scale tabular foundation models.

Figures (4)

• Figure 1: Overview of the proposed CTSyn framework.
• Figure 2: Downstream Machine Learning Utility on Classification and Regrssion Datasets, on synthetic data from different generators.
• Figure 3: T-sne plot of Indian Liver Patient dataset from different synthesizers.
• Figure 4: Tsne visualization of embedding space created by pre-trained VAE model, on columns of Insurance dataset.