Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Reinforcement Learning Approach to Synthetic Data Generation

Natalia Espinosa-Dice, Nicholas J. Jackson, Chao Yan, Aaron Lee, Bradley A. Malin

TL;DR

This work reframes synthetic data generation (SDG) as a reinforcement learning problem and introduces RLSyn, a PPO-based framework where the generator is a stochastic policy over full patient records trained with discriminator-derived rewards. Evaluated on AI-READI and MIMIC-IV against GAN and diffusion baselines, RLSyn achieves competitive performance with diffusion models and outperforms GANs, particularly in small-sample regimes, while maintaining privacy. The results demonstrate that reinforcement learning can provide a principled and data-efficient alternative for privacy-preserving SDG in biomedicine, with strong utility and fidelity in data-scarce settings. Overall, RLSyn represents a promising direction for sharing synthetic biomedical data without compromising patient privacy, and opens avenues for extending RL-based SDG to other modalities and reward formulations.

Abstract

Synthetic data generation (SDG) is a promising approach for enabling data sharing in biomedical studies while preserving patient privacy. Yet, state-of-the-art generative models often require large datasets and complex training procedures, limiting their applicability in small-sample settings. In this work, we reframe SDG as a reinforcement learning (RL) problem and introduce RLSyn, a novel framework that models the data generator as a stochastic policy over patient records and optimizes it using Proximal Policy Optimization with discriminator-derived rewards, yielding more stable and data-efficient training. We evaluate RLSyn on two biomedical datasets - AI-READI and MIMIC-IV- and benchmark it against state-of-the-art generative adversarial networks (GANs) and diffusion-based methods across extensive privacy, utility, and fidelity evaluations. RL-Syn performs comparably to diffusion models and outperforms GANs on MIMIC-IV, while outperforming both diffusion models and GANs on the smaller AI-READI dataset. These results demonstrate that reinforcement learning provides a principled and effective alternative for synthetic biomedical data generation, particularly in data-scarce regimes.

A Reinforcement Learning Approach to Synthetic Data Generation

TL;DR

This work reframes synthetic data generation (SDG) as a reinforcement learning problem and introduces RLSyn, a PPO-based framework where the generator is a stochastic policy over full patient records trained with discriminator-derived rewards. Evaluated on AI-READI and MIMIC-IV against GAN and diffusion baselines, RLSyn achieves competitive performance with diffusion models and outperforms GANs, particularly in small-sample regimes, while maintaining privacy. The results demonstrate that reinforcement learning can provide a principled and data-efficient alternative for privacy-preserving SDG in biomedicine, with strong utility and fidelity in data-scarce settings. Overall, RLSyn represents a promising direction for sharing synthetic biomedical data without compromising patient privacy, and opens avenues for extending RL-based SDG to other modalities and reward formulations.

Abstract

Synthetic data generation (SDG) is a promising approach for enabling data sharing in biomedical studies while preserving patient privacy. Yet, state-of-the-art generative models often require large datasets and complex training procedures, limiting their applicability in small-sample settings. In this work, we reframe SDG as a reinforcement learning (RL) problem and introduce RLSyn, a novel framework that models the data generator as a stochastic policy over patient records and optimizes it using Proximal Policy Optimization with discriminator-derived rewards, yielding more stable and data-efficient training. We evaluate RLSyn on two biomedical datasets - AI-READI and MIMIC-IV- and benchmark it against state-of-the-art generative adversarial networks (GANs) and diffusion-based methods across extensive privacy, utility, and fidelity evaluations. RL-Syn performs comparably to diffusion models and outperforms GANs on MIMIC-IV, while outperforming both diffusion models and GANs on the smaller AI-READI dataset. These results demonstrate that reinforcement learning provides a principled and effective alternative for synthetic biomedical data generation, particularly in data-scarce regimes.
Paper Structure (22 sections, 4 figures, 3 tables, 2 algorithms)

This paper contains 22 sections, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Statement of Significance
  3. Related Work
  4. Generative Adversarial Networks
  5. Denoising Diffusion Models
  6. Reinforcement Learning
  7. Methodology
  8. Datasets
  9. RLSyn
  10. Baselines
  11. Evaluation Criteria
  12. Privacy
  13. Utility
  14. Fidelity
  15. Model Training
  16. ...and 7 more sections

Figures (4)

  • Figure 1: RL-SYN Generator Architecture. A shared multilayer perceptron (MLP) maps a latent noise vector to three output heads: a categorical head that parameterizes a distribution over discrete features, a continuous head that parameterizes a distribution over continuous features, and a value head that estimates the expected reward of the generated record. The categorical and continuous outputs are combined to form a synthetic record $x$, whose log-probability under the generator and associated value estimate are used to update the generator during training.
  • Figure 2: A comparison of the performance of classifiers trained on synthetic data and tested on the real test dataset (S2R) and vice versa (R2S) for both the MIMIC-IV and AI-READI datasets. The dashed black line represents the baseline of classifiers trained and tested on real data (R2R). Error bars represent 95% Confidence Intervals.
  • Figure 3: A scatterplot of the first two principal components of the real (red) and synthetic (blue) data.
  • Figure 4: A scatterplot of the first two principle components for real (red) vs synthetic (blue) data.