Differentially Private Quasi-Concave Optimization: Bypassing the Lower Bound and Application to Geometric Problems

TL;DR

The paper addresses the sample complexity of differentially private optimization of quasi-concave functions, showing that the previously established $Ω(2^{\log^*|\mathcal{X}|})$ lower bound can be bypassed for natural problem classes by introducing approximated quasi-concave functions. It introduces the IPConcave framework, which reduces optimization to 1D interior-point methods and extends to higher dimensions with coordinate-wise updates, achieving $\tilde{O}(d^{5.5}\cdot\log^*|\mathcal{X}|)$ sample complexity for privately selecting a center point and privately learning $d$-dimensional halfspaces. A key insight is that VC theory enables small-size approximations of depth-like targets (e.g., Tukey depth and cdepth) to enable private optimization under DP, yielding privacy-preserving center points and halfspace learners with strong domain-size guarantees. Furthermore, the results imply that any VC dimension-1 class can be privately PAC-learned with $\tilde{O}(\log^*|\mathcal{X}|)$ samples, highlighting a fundamental improvement in the DP sample complexity landscape for these geometric tasks. Open questions remain regarding reducing the dependence on dimension $d$ while preserving the improved $\log^*|\mathcal{X}|$-type dependence on the domain size.

Abstract

We study the sample complexity of differentially private optimization of quasi-concave functions. For a fixed input domain $\mathcal{X}$, Cohen et al. (STOC 2023) proved that any generic private optimizer for low sensitive quasi-concave functions must have sample complexity $Ω(2^{\log^*|\mathcal{X}|})$. We show that the lower bound can be bypassed for a series of ``natural'' problems. We define a new class of \emph{approximated} quasi-concave functions, and present a generic differentially private optimizer for approximated quasi-concave functions with sample complexity $\tilde{O}(\log^*|\mathcal{X}|)$. As applications, we use our optimizer to privately select a center point of points in $d$ dimensions and \emph{probably approximately correct} (PAC) learn $d$-dimensional halfspaces. In previous works, Bun et al. (FOCS 2015) proved a lower bound of $Ω(\log^*|\mathcal{X}|)$ for both problems. Beimel et al. (COLT 2019) and Kaplan et al. (NeurIPS 2020) gave an upper bound of $\tilde{O}(d^{2.5}\cdot 2^{\log^*|\mathcal{X}|})$ for the two problems, respectively. We improve the dependency of the upper bounds on the cardinality of the domain by presenting a new upper bound of $\tilde{O}(d^{5.5}\cdot\log^*|\mathcal{X}|)$ for both problems. To the best of our understanding, this is the first work to reduce the sample complexity dependency on $|\mathcal{X}|$ for these two problems from exponential in $\log^* |\mathcal{X}|$ to $\log^* |\mathcal{X}|$.

Figures (2)

• Figure 1: Depth and cdepth
• Figure 2: 2 halfspaces can be shattered (left) and 3 halfspaces cannot be shattered (right) in the 2-D plane

Theorems & Definitions (52)

• Definition 1: Differential Privacy DMNS06
• Theorem 1: Informal
• Theorem 2: Informal
• Theorem 3: Privately approximating the center point
• Theorem 4: Privately learning halfspaces
• Corollary 1
• Definition 2: Vapnik-Chervonenkis dimension VCHaussler1986EpsilonnetsAS
• Definition 3: $\alpha$-approximation VCHaussler1986EpsilonnetsAS
• Theorem 5: VCHaussler1986EpsilonnetsAS
• Definition 4: Generalization and empirical error