Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Note on Statistically Accurate Tabular Data Generation Using Large Language Models

Andrey Sidorenko

TL;DR

The paper tackles the challenge of generating realistic tabular data with large language models while preserving complex feature dependencies, especially for categorical variables. It introduces a probability-driven prompting approach that first infers conditional distributions and then samples from these distributions to create synthetic rows, reducing reliance on token-by-token generation. In a California demographic case study, the method demonstrates improved statistical fidelity over table-wide and cell-by-cell prompting, while remaining computationally scalable with a fixed set of distributional prompts. The approach offers a practical, adaptable solution for privacy-preserving, high-volume tabular data synthesis.

Abstract

Large language models (LLMs) have shown promise in synthetic tabular data generation, yet existing methods struggle to preserve complex feature dependencies, particularly among categorical variables. This work introduces a probability-driven prompting approach that leverages LLMs to estimate conditional distributions, enabling more accurate and scalable data synthesis. The results highlight the potential of prompting probability distributions to enhance the statistical fidelity of LLM-generated tabular data.

A Note on Statistically Accurate Tabular Data Generation Using Large Language Models

TL;DR

The paper tackles the challenge of generating realistic tabular data with large language models while preserving complex feature dependencies, especially for categorical variables. It introduces a probability-driven prompting approach that first infers conditional distributions and then samples from these distributions to create synthetic rows, reducing reliance on token-by-token generation. In a California demographic case study, the method demonstrates improved statistical fidelity over table-wide and cell-by-cell prompting, while remaining computationally scalable with a fixed set of distributional prompts. The approach offers a practical, adaptable solution for privacy-preserving, high-volume tabular data synthesis.

Abstract

Large language models (LLMs) have shown promise in synthetic tabular data generation, yet existing methods struggle to preserve complex feature dependencies, particularly among categorical variables. This work introduces a probability-driven prompting approach that leverages LLMs to estimate conditional distributions, enabling more accurate and scalable data synthesis. The results highlight the potential of prompting probability distributions to enhance the statistical fidelity of LLM-generated tabular data.
Paper Structure (7 sections, 1 equation, 1 figure, 1 table)

This paper contains 7 sections, 1 equation, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Results and Discussion
  4. Summary
  5. Population distribution in California, US.
  6. Prompts
  7. Pseudo-code for probability-based distribution generation

Figures (1)

  • Figure 1: Comparison of ethnic composition by age group in California across real and synthetic datasets. (a) Empirical distribution from the US Census Bureau (July 1, 2023) ppic2023population; (b) Synthetic distribution using a table-wide generation approach; (c) Synthetic distribution via a cell-by-cell generation method; (d) Synthetic distribution using a probability-driven prompting approach (this work). The probability-driven prompting model preserves age-dependent demographic heterogeneity most correctly relative to the real distribution, outperforming traditional generation strategies in both accuracy and diversity of representation.