CTTVAE: Latent Space Structuring for Conditional Tabular Data Generation on Imbalanced Datasets

TL;DR

CTTVAE addresses severe class imbalance in tabular data by combining a conditional transformer-based VAE with latent-space structuring via a class-aware triplet margin loss and a training-by-sampling strategy. The model performs conditional generation confined to class-specific latent regions using local triangle interpolation, while TBS ensures underrepresented groups receive more training exposure. Ablation studies show both latent structuring and targeted sampling contribute to improved minority-class utility, with CTTVAE+TBS achieving strong performance while maintaining competitive fidelity and privacy. Across six real-world benchmarks, CTTVAE+TBS consistently enhances minority utility (downstream predictive performance) without sacrificing majority-class performance, offering a practical, privacy-conscious approach for sensitive domains like healthcare and fraud detection. The work demonstrates a favorable utility–privacy–fidelity trade-off and provides a configurable framework for conditional tabular data generation under imbalance.

Abstract

Generating synthetic tabular data under severe class imbalance is essential for domains where rare but high-impact events drive decision-making. However, most generative models either overlook minority groups or fail to produce samples that are useful for downstream learning. We introduce CTTVAE, a Conditional Transformer-based Tabular Variational Autoencoder equipped with two complementary mechanisms: (i) a class-aware triplet margin loss that restructures the latent space for sharper intra-class compactness and inter-class separation, and (ii) a training-by-sampling strategy that adaptively increases exposure to underrepresented groups. Together, these components form CTTVAE+TBS, a framework that consistently yields more representative and utility-aligned samples without destabilizing training. Across six real-world benchmarks, CTTVAE+TBS achieves the strongest downstream utility on minority classes, often surpassing models trained on the original imbalanced data while maintaining competitive fidelity and bridging the gap for privacy for interpolation-based sampling methods and deep generative methods. Ablation studies further confirm that both latent structuring and targeted sampling contribute to these gains. By explicitly prioritizing downstream performance in rare categories, CTTVAE+TBS provides a robust and interpretable solution for conditional tabular data generation, with direct applicability to industries such as healthcare, fraud detection, and predictive maintenance where even small gains in minority cases can be critical.

Figures (7)

• Figure 1: Conditional generation with CTTVAE. (1) The trained encoder maps real samples to latent representations $\mathbf{z}$, from which class-specific subsets are retained. (2) A filtration step isolates the latent region corresponding to the target class ($\mathcal{S}_c$). (3) Synthetic latent points $\hat{\mathbf{z}}$ are generated by local triangle interpolation: for each anchor latent point $z_i$, sampling occurs within a local convex region spanned by its $k = 5$ nearest neighbors, using inverse-rank distance weighting and per-neighbor random scaling (bottom zoom panel). The decoder then reconstructs synthetic samples from $(\hat{\mathbf{z}}, \mathbf{h})$.
• Figure 2: The absolute difference between correlation matrices computed on real and synthetic datasets. More intense red color indicates higher difference. Among deep generative methods, TabDiff and our methods capture correlations better.
• Figure 3: Impact on the minority class of the sampling hyperparameter $\lambda$ on F1 scores across datasets for CTTVAE+TBS and TTVAE+TBS. $\lambda=1$ represents the models performances without aplying TBS. Performance on minority classes depends greatly on its value.
• Figure 4: Pipeline
• Figure 5: Minority-class trade-offs between utility and (a) privacy and (b) fidelity. The $x$–axis shows the geometric mean ratio of minority CatBoost MLE on synthetic vs. real data (higher is better). The $y$–axis reports the corresponding ratio for minority NNDR (higher is better) or WD (lower is better). The point labelled Optimal is a conceptual reference corresponding to matching real-data utility with perfect privacy or zero fidelity error. CTTVAE, TTVAE+TBS and CTTVAE+TBS all lie in the desirable high-utility region, with CTTVAE+TBS achieving the best minority utility while retaining competitive privacy and fidelity compared to both classical GAN/VAE baselines and the diffusion-based TabDiff.