Cascaded Flow Matching for Heterogeneous Tabular Data with Mixed-Type Features

TL;DR

This work tackles the challenge of generating heterogeneous tabular data with mixed-type features by introducing TabCascade, a cascaded flow matching framework that separately learns a low-resolution, categorical representation and a high-resolution numerical refinement. A novel guided conditional probability path and data-dependent coupling redirect the learning focus to learning fine-grained details while leveraging coarse structure, and a low-resolution encoder (DT or GMM) provides discrete latents that capture missing and inflated states. The authors prove that this cascade tightens the transport cost bound and show through extensive experiments that TabCascade achieves higher joint distribution realism, improved feature-wise fidelity, and enhanced downstream-task utility, with meaningful ablations supporting the value of the core components. The approach offers a practical path to realistic synthesis of mixed-type tabular data, with implications for data imputation, privacy considerations, and potential extensions to other modalities. Overall, TabCascade advances tabular data synthesis by effectively handling mixed-type features and missing values through a principled cascaded, conditional generation strategy.

Abstract

Advances in generative modeling have recently been adapted to tabular data containing discrete and continuous features. However, generating mixed-type features that combine discrete states with an otherwise continuous distribution in a single feature remains challenging. We advance the state-of-the-art in diffusion models for tabular data with a cascaded approach. We first generate a low-resolution version of a tabular data row, that is, the collection of the purely categorical features and a coarse categorical representation of numerical features. Next, this information is leveraged in the high-resolution flow matching model via a novel guided conditional probability path and data-dependent coupling. The low-resolution representation of numerical features explicitly accounts for discrete outcomes, such as missing or inflated values, and therewith enables a more faithful generation of mixed-type features. We formally prove that this cascade tightens the transport cost bound. The results indicate that our model generates significantly more realistic samples and captures distributional details more accurately, for example, the detection score increases by 40%.

Figures (25)

• Figure 1: Overview of TabCascade for the missing value generation task. We derive a categorical, low-resolution representation ${\mathbf{z}}$ from ${\mathbf{x}}_{\text{num}}$, form ${\mathbf{x}}_{\text{low}} = ({\mathbf{x}}_{\text{cat}}, {\mathbf{z}})$ and then learn $p_{\text{low}}({\mathbf{x}}_{\text{low}})$. We then learn the high-resolution distribution $p_{\text{high}}({\mathbf{x}}_{\text{num}} | {\mathbf{x}}_{\text{low}})$ conditional on ${\mathbf{x}}_{\text{low}}$. This reduces the transport cost bound and simplifies the learning task. The discrete state ${\mathbf{z}}$ enables the model to naturally handle mixed-type feature distributions at generation time. This approach generalizes to arbitrary (and multiple) discrete states.
• Figure 2: Motivational results
• Figure 3: Densities $p_t$ generated from (top) a flow model with data-dependent coupling and non-linear paths, and (bottom) a classic flow model with linear paths and independent coupling. Both models condition on ${\mathbf{z}}$. For the top model, the source distribution is $p_0({\mathbf{x}}_0) = \int p({\mathbf{x}}_0, {\mathbf{x}}_1) \mathrm{d} {\mathbf{x}}_1$ with $p({\mathbf{x}}_0, {\mathbf{x}}_1)$ as defined in \ref{['eq:data_dependent_coupling']}. WD represents the Wasserstein distance of $p_t$ to the true data distribution. The data-dependent coupling induces a source distribution that is much closer to the data distribution, and thus effectively reduces transport costs. Savings in model capacity and time are spent on more efficient learning of distributional details.
• Figure 4: Gaussian components found by the GMM encoder (max components = 7, to align with the number of components found by DT) for two features in the adult dataset. The red vertical lines indicate the means of the Gaussian components.
• Figure 5: Gaussian components found by the DT encoder (max depth = 3) for two features in the adult dataset. The red vertical lines indicate the means of the Gaussian components.

Theorems & Definitions (2)

• Theorem 1: Data-dependent coupling tightens transport cost bound
• proof