Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Two-Stage Data Synthesization: A Statistics-Driven Restricted Trade-off between Privacy and Prediction

Xiaotong Liu, Shao-Bo Lin, Jun Fan, Ding-Xuan Zhou

TL;DR

This work addresses the challenge of generating anonymized data that preserves predictive performance while defending against privacy attacks. It introduces a two-stage LK-2SS framework that first uses a synthesis-then-hybrid strategy to retain covariant input distributions, then applies a kernel ridge regression model to reconstruct outputs from synthetic inputs, yielding a statistics-driven restricted privacy–prediction trade-off. The approach is supported by theoretical guarantees under an integral-operator formalism, demonstrating bounds that trade off total-variation distance and estimation error, and is validated on a marketing price–sale task and five real-world datasets with multi-scenario evaluations showing robustness to distribution shifts and limited external data. Collectively, LK-2SS offers a practical, theory-backed pathway to generate high-quality synthetic data suitable for real-world prediction tasks while maintaining controlled privacy leakage via the LID framework and distributional control via the two-stage design.

Abstract

Synthetic data have gained increasing attention across various domains, with a growing emphasis on their performance in downstream prediction tasks. However, most existing synthesis strategies focus on maintaining statistical information. Although some studies address prediction performance guarantees, their single-stage synthesis designs make it challenging to balance the privacy requirements that necessitate significant perturbations and the prediction performance that is sensitive to such perturbations. We propose a two-stage synthesis strategy. In the first stage, we introduce a synthesis-then-hybrid strategy, which involves a synthesis operation to generate pure synthetic data, followed by a hybrid operation that fuses the synthetic data with the original data. In the second stage, we present a kernel ridge regression (KRR)-based synthesis strategy, where a KRR model is first trained on the original data and then used to generate synthetic outputs based on the synthetic inputs produced in the first stage. By leveraging the theoretical strengths of KRR and the covariant distribution retention achieved in the first stage, our proposed two-stage synthesis strategy enables a statistics-driven restricted privacy--prediction trade-off and guarantee optimal prediction performance. We validate our approach and demonstrate its characteristics of being statistics-driven and restricted in achieving the privacy--prediction trade-off both theoretically and numerically. Additionally, we showcase its generalizability through applications to a marketing problem and five real-world datasets.

Two-Stage Data Synthesization: A Statistics-Driven Restricted Trade-off between Privacy and Prediction

TL;DR

This work addresses the challenge of generating anonymized data that preserves predictive performance while defending against privacy attacks. It introduces a two-stage LK-2SS framework that first uses a synthesis-then-hybrid strategy to retain covariant input distributions, then applies a kernel ridge regression model to reconstruct outputs from synthetic inputs, yielding a statistics-driven restricted privacy–prediction trade-off. The approach is supported by theoretical guarantees under an integral-operator formalism, demonstrating bounds that trade off total-variation distance and estimation error, and is validated on a marketing price–sale task and five real-world datasets with multi-scenario evaluations showing robustness to distribution shifts and limited external data. Collectively, LK-2SS offers a practical, theory-backed pathway to generate high-quality synthetic data suitable for real-world prediction tasks while maintaining controlled privacy leakage via the LID framework and distributional control via the two-stage design.

Abstract

Synthetic data have gained increasing attention across various domains, with a growing emphasis on their performance in downstream prediction tasks. However, most existing synthesis strategies focus on maintaining statistical information. Although some studies address prediction performance guarantees, their single-stage synthesis designs make it challenging to balance the privacy requirements that necessitate significant perturbations and the prediction performance that is sensitive to such perturbations. We propose a two-stage synthesis strategy. In the first stage, we introduce a synthesis-then-hybrid strategy, which involves a synthesis operation to generate pure synthetic data, followed by a hybrid operation that fuses the synthetic data with the original data. In the second stage, we present a kernel ridge regression (KRR)-based synthesis strategy, where a KRR model is first trained on the original data and then used to generate synthetic outputs based on the synthetic inputs produced in the first stage. By leveraging the theoretical strengths of KRR and the covariant distribution retention achieved in the first stage, our proposed two-stage synthesis strategy enables a statistics-driven restricted privacy--prediction trade-off and guarantee optimal prediction performance. We validate our approach and demonstrate its characteristics of being statistics-driven and restricted in achieving the privacy--prediction trade-off both theoretically and numerically. Additionally, we showcase its generalizability through applications to a marketing problem and five real-world datasets.
Paper Structure (34 sections, 9 theorems, 56 equations, 13 figures, 8 tables)

This paper contains 34 sections, 9 theorems, 56 equations, 13 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Organization
  4. Problem Setting and Related Work
  5. Privacy and prediction issues in data synthesization
  6. Problem setting
  7. Related work
  8. Two-Stage Data Synthesization
  9. Covariant distribution retention: synthetic-then-hybrid strategy
  10. Response reconstruction: KRR-based synthesis strategy
  11. Statistics retention strategy
  12. Statistics-driven two-stage synthesis strategy
  13. Theoretical Verification
  14. Statistics-Based Restricted Privacy–Prediction Trade-off
  15. Power of LK-2SS Strategy
  16. ...and 19 more sections

Key Result

theorem 1

Let $\delta\in(0,1)$, $\epsilon>0$ be an arbitrarily small positive number, $\rho^*_X$ be a synthesization distribution and $\rho_{X,\alpha}^*=(1-\alpha)\rho_X^*+\alpha\rho_X$ be the hybrid distribution. If Assumption Assumption:regularity_and_effective_dimension holds with $\frac{1}{2}\leq r\leq 1$ where $C'$ is a constant depending only on $C_0,M,r,s,\kappa$ and $\|h^\star\|_\rho$. If in additio

Figures (13)

  • Figure 1: Synthesis strategies: timeline, focused trade-offs, and practical applications (presenting only a few common strategies as discussed in lu2023machinefigueira2022survey).
  • Figure 2: Comparison of different privacy principles
  • Figure 3: Two-stage synthesis strategy
  • Figure 4: Synthesis–hybrid trade-off in synthesis-then-hybrid strategy
  • Figure 5: Statistics-based restricted privacy–prediction trade-off enabled by the synthesis-then-hybrid strategy
  • ...and 8 more figures

Theorems & Definitions (10)

  • definition 1: Location-based interval disclosure
  • theorem 1
  • lemma 1
  • proposition 1
  • lemma 2
  • lemma 3
  • lemma 4
  • lemma 5
  • proposition 2
  • lemma 6