Differentially Private Sliced Inverse Regression: Minimax Optimality and Algorithm

TL;DR

The paper tackles differentially private sufficient dimension reduction by analyzing minimax limits and designing DP algorithms for SIR in both low- and high-dimensional settings. It introduces a DP histogram approach to privately slice responses, a gradient-based estimator with a peeling step for sparsity, and DP initializers, achieving near-optimal rates up to logarithmic factors. Theoretical results provide matching lower and upper bounds under standard SDR and DP assumptions, while simulations and a real-world supermarket dataset demonstrate practical privacy-utility trade-offs. Overall, the work offers a principled, scalable framework for privacy-preserving dimension reduction with potential extensions to DP sparse PCA and related methods.

Abstract

Privacy preservation has become a critical concern in high-dimensional data analysis due to the growing prevalence of data-driven applications. Since its proposal, sliced inverse regression has emerged as a widely utilized statistical technique to reduce the dimensionality of covariates while maintaining sufficient statistical information. In this paper, we propose optimally differentially private algorithms specifically designed to address privacy concerns in the context of sufficient dimension reduction. We establish lower bounds for differentially private sliced inverse regression in low and high dimensional settings. Moreover, we develop differentially private algorithms that achieve the minimax lower bounds up to logarithmic factors. Through a combination of simulations and real data analysis, we illustrate the efficacy of these differentially private algorithms in safeguarding privacy while preserving vital information within the reduced dimension space. As a natural extension, we can readily offer analogous lower and upper bounds for differentially private sparse principal component analysis, a topic that may also be of potential interest to the statistics and machine learning community.

Figures (14)

• Figure 1: The scatter plot between the response $Y$ and $\widehat{X}$ which is the projection of $\boldsymbol{x}$. The black solid curve is the fitted spline regression curve, and the gray shaded areas are corresponding confidence regions.
• Figure 2: The scatter plots between the response $Y$ and $X_5$ and $X_{23}$. The black solid curves are the fitted spline regression curves for three variables, respectively, and the gray shaded areas are corresponding confidence regions.
• Figure C.1: The ROC curves of the DP-SIni, DP-TRF, DP-SSIR, and Lasso-SIR.
• Figure C.2: Comparison of the eigen-gaps between the kernel matrix based on sample quantiles and that based on DP estimated slices for privacy parameter $\varepsilon=0.1$ under two generation models and varying sample sizes. The plot uses $m=\lceil 8n^{1/3}\rceil$ bins and $H=10$ slices.
• Figure C.3: Comparison of the loss in the estimated matrix $\boldsymbol{B}$ derived from the kernel matrix based on sample quantiles versus that based on DP estimated slices for privacy parameter $\varepsilon=0.1$ under two generation models and varying sample size. The plot uses $m=\lceil 8n^{1/3}\rceil$ bins and $H=10$ slices.

Theorems & Definitions (50)

• Definition 1: Differential Privacy dwork2006calibrating
• Definition 2: Sensitivity
• Lemma 2.1: Extended from Theorem 3 in dwork2021differentially
• Theorem 1
• Lemma 3.1
• Lemma 3.2: Privacy Guarantee of Algorithm \ref{['alg:ld']}
• Theorem 2: Convergence of Algorithm \ref{['alg:ld']}
• Theorem 3
• Theorem 4
• Lemma 4.1: Privacy Guarantee of Algorithm \ref{['alg:hd_dp_sir']}