Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Technical Exploration of Causal Inference with Hybrid LLM Synthetic Data

Dana Kim, Yichen Xu, Tiffany Lin

TL;DR

This paper investigates preserving causal parameters, particularly the average treatment effect (ATE) defined as $\Psi(P) = \mathbb{E}_P[ \mathbb{E}_P[Y \mid A=1,W] - \mathbb{E}_P[Y \mid A=0,W] ]$, when generating synthetic tabular data with Large Language Models. It proposes a hybrid data-generation framework that synthesizes covariates with a generative model while separately learning a propensity score model $\hat{g}(A \mid W)$ and an outcome model $\hat{h}(Y \mid A,W)$, then samples $\tilde{A}$ and $\tilde{Y}$ to form triplets $(\tilde{W},\tilde{A},\tilde{Y})$ that preserve causal structure. An evaluation protocol uses unlimited synthetic samples to benchmark causal estimators (IPTW, AIPW, substitution) under realistic covariate distributions, showing the hybrid approach yields more accurate ATE estimates than fully synthetic methods. Real-world NHANES data illustrate the necessity of careful model tuning and demonstrate how synthetic pairing and DCR-based filtering can improve the reliability of causal inferences in practice.

Abstract

Large Language Models (LLMs) offer a flexible means to generate synthetic tabular data, yet existing approaches often fail to preserve key causal parameters such as the average treatment effect (ATE). In this technical exploration, we first demonstrate that state-of-the-art synthetic data generators, both GAN- and LLM-based, can achieve high predictive fidelity while substantially misestimating causal effects. To address this gap, we propose a hybrid generation framework that combines model-based covariate synthesis (monitored via distance-to-closest-record filtering) with separately learned propensity and outcome models, thereby ensuring that (W, A, Y) triplets retain their underlying causal structure. We further introduce a synthetic pairing strategy to mitigate positivity violations and a realistic evaluation protocol that leverages unlimited synthetic samples to benchmark traditional estimators (IPTW, AIPW, substitution) under complex covariate distributions. This work lays the groundwork for LLM-powered data pipelines that support robust causal analysis. Our code is available at https://github.com/Xyc-arch/llm-synthetic-for-causal-inference.git.

A Technical Exploration of Causal Inference with Hybrid LLM Synthetic Data

TL;DR

This paper investigates preserving causal parameters, particularly the average treatment effect (ATE) defined as , when generating synthetic tabular data with Large Language Models. It proposes a hybrid data-generation framework that synthesizes covariates with a generative model while separately learning a propensity score model and an outcome model , then samples and to form triplets that preserve causal structure. An evaluation protocol uses unlimited synthetic samples to benchmark causal estimators (IPTW, AIPW, substitution) under realistic covariate distributions, showing the hybrid approach yields more accurate ATE estimates than fully synthetic methods. Real-world NHANES data illustrate the necessity of careful model tuning and demonstrate how synthetic pairing and DCR-based filtering can improve the reliability of causal inferences in practice.

Abstract

Large Language Models (LLMs) offer a flexible means to generate synthetic tabular data, yet existing approaches often fail to preserve key causal parameters such as the average treatment effect (ATE). In this technical exploration, we first demonstrate that state-of-the-art synthetic data generators, both GAN- and LLM-based, can achieve high predictive fidelity while substantially misestimating causal effects. To address this gap, we propose a hybrid generation framework that combines model-based covariate synthesis (monitored via distance-to-closest-record filtering) with separately learned propensity and outcome models, thereby ensuring that (W, A, Y) triplets retain their underlying causal structure. We further introduce a synthetic pairing strategy to mitigate positivity violations and a realistic evaluation protocol that leverages unlimited synthetic samples to benchmark traditional estimators (IPTW, AIPW, substitution) under complex covariate distributions. This work lays the groundwork for LLM-powered data pipelines that support robust causal analysis. Our code is available at https://github.com/Xyc-arch/llm-synthetic-for-causal-inference.git.

Paper Structure

This paper contains 14 sections, 1 equation, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Using LLMs as Realistic Synthetic Data Generators
  4. Textual Serialization and Prompt Construction
  5. Fine-Tuning with GReaT
  6. Sampling Strategy
  7. Advantages and Fidelity Considerations
  8. Runtime and Reproducibility
  9. Failure of Generative Model to Preserve Causal Parameter and A Hybrid generation approach
  10. A valid way to address positivity with LLM synthetic data
  11. LLM Synthetic data for realistic estimator evaluation
  12. Real-World Applications and Limitations
  13. Conclusion
  14. GitHub, Video, Slides, Info

Figures (4)

  • Figure 1: Overview of the LLM‐based synthetic data generation pipeline. Raw data is first serialized into prompts and processed through an LLM. The resulting synthetic samples are filtered via DCR (distance to closest record) matching. Curated synthetic data is evaluated for privacy preservation, positivity balance, and estimator fidelity before contributing to ATE estimation.
  • Figure 2: Boxplot of Distance to Closest Record (DCR).
  • Figure 3: Bar plots of ATE IPTW estimate by different baselines.
  • Figure 4: Bar plots of ATE AIPW estimate by different baselines.