EPIC: Effective Prompting for Imbalanced-Class Data Synthesis in Tabular Data Classification via Large Language Models

TL;DR

The paper tackles generating high-quality synthetic tabular data for imbalanced classification by leveraging in-context learning with large language models. It introduces EPIC, a prompting framework that uses CSV-style data, balanced class grouping, and a unique variable mapping to guide LLMs to produce realistic, class-balanced samples without any parameter updates. Through extensive experiments on six real-world datasets and analysis of prompt design and feature correlations, EPIC achieves state-of-the-art classification performance and improved generation efficiency, particularly enhancing minority-class representation. The approach demonstrates strong results with both closed- and open-source LLMs, suggesting broad practical applicability and potential privacy benefits by reducing dependence on real data. Overall, EPIC provides a robust, scalable solution for synthetic tabular data generation that preserves feature relationships and class distributions, addressing a key bottleneck in imbalanced data ML applications.

Abstract

Large language models (LLMs) have demonstrated remarkable in-context learning capabilities across diverse applications. In this work, we explore the effectiveness of LLMs for generating realistic synthetic tabular data, identifying key prompt design elements to optimize performance. We introduce EPIC, a novel approach that leverages balanced, grouped data samples and consistent formatting with unique variable mapping to guide LLMs in generating accurate synthetic data across all classes, even for imbalanced datasets. Evaluations on real-world datasets show that EPIC achieves state-of-the-art machine learning classification performance, significantly improving generation efficiency. These findings highlight the effectiveness of EPIC for synthetic tabular data generation, particularly in addressing class imbalance. Our source code for our work is available at: https://seharanul17.github.io/project-synthetic-tabular-llm/

Figures (14)

• Figure 1: Generation results on an imbalanced toy dataset with majority and minority classes. Our approach, leveraging in-context learning with LLMs, achieves (1) distinct class boundaries, (2) accurate feature correlations, (3) well-matched value ranges, (4) robust numerical-categorical relationships (last row), and (5) comprehensive data distribution coverage, with improvements over its ablated versions and the fine-tuned GReaT model borisov2022language_great. Complete results are available in Fig. \ref{['appen_fig:toy']}.
• Figure 2: Comparison of ML classification performance with synthetic data on the Travel dataset. Results are averaged across four classifiers: XGBoost, CatBoost, LightGBM, and gradient boosting classifier, with each classifier run five times. Our method uses the GPT-3.5-turbo model.
• Figure 3: Overview of our approach. Our prompt includes repeated data example sets consisting of feature names and class-balanced groups, with the feature name at the end serving as a trigger for the LLM to generate realistic synthetic tabular data. The proposed unique variable remaps categorical values to distinct alphanumeric strings, ensuring clear distinction and variability among variables.
• Figure 4: Difference between Cramér's V correlation matrices of real and synthetic datasets for categorical variables in the Travel dataset. More intense colors indicate larger differences, with positive differences shown in red and negative differences shown in blue. Black indicates where the correlation is not measured since only one unique value is generated for those variables.
• Figure 5: Difference between Pearson correlation matrices of real and synthetic data for numerical variables in the Sick dataset. More intense colors indicate larger differences, with positive differences shown in red and negative differences shown in blue. Black indicates where the correlation is not measured since only one unique value is generated for those variables.