Sparse Point-wise Privacy Leakage: Mechanism Design and Fundamental Limits

TL;DR

The paper tackles privacy mechanism design when disclosed data $U$ is generated from $Y$ while a sensitive variable $X$ is only indirectly correlated with $Y$. It introduces sparse point-wise privacy leakage, enforcing per-output constraints that limit both the number of $X$-realizations a symbol $u$ can inform ($N$) and the total leakage via a $\chi^2$-divergence budget. In the high-privacy regime, a local information-geometry expansion yields a quadratic approximation of $I(U;Y)$, and with an invertible $P_{X|Y}$ this reduces to a sparse Rayleigh-quotient optimization on the subspace orthogonal to $\sqrt{P_X}$ that, surprisingly, is solved by binary, uniform $U$. The problem is NP-hard in general and connected to sparse PCA, motivating a convex SDP relaxation with a rounding step and revealing a deterministic sparsity threshold beyond which the relaxation is tight. Numerical experiments show a sharp phase transition where the SDP solution matches the exact optimum, suggesting a practical, polynomial-time pathway to near-optimal sparse leakage designs while guaranteeing privacy-utility trade-offs.

Abstract

We study an information-theoretic privacy mechanism design problem, where an agent observes useful data $Y$ that is arbitrarily correlated with sensitive data $X$, and design disclosed data $U$ generated from $Y$ (the agent has no direct access to $X$). We introduce \emph{sparse point-wise privacy leakage}, a worst-case privacy criterion that enforces two simultaneous constraints for every disclosed symbol $u\in\mathcal{U}$: (i) $u$ may be correlated with at most $N$ realizations of $X$, and (ii) the total leakage toward those realizations is bounded. In the high-privacy regime, we use concepts from information geometry to obtain a local quadratic approximation of mutual information which measures utility between $U$ and $Y$. When the leakage matrix $P_{X|Y}$ is invertible, this approximation reduces the design problem to a sparse quadratic maximization, known as the Rayleigh-quotient problem, with an $\ell_0$ constraint. We further show that, for the approximated problem, one can without loss of optimality restrict attention to a binary released variable $U$ with a uniform distribution. For small alphabet sizes, the exact sparsity-constrained optimum can be computed via combinatorial support enumeration, which quickly becomes intractable as the dimension grows. For general dimensions, the resulting sparse Rayleigh-quotient maximization is NP-hard and closely related to sparse principal component analysis (PCA). We propose a convex semidefinite programming (SDP) relaxation that is solvable in polynomial time and provides a tractable surrogate for the NP-hard design, together with a simple rounding procedure to recover a feasible leakage direction. We also identify a sparsity threshold beyond which the sparse optimum saturates at the unconstrained spectral value and the SDP relaxation becomes tight.

Figures (2)

• Figure 1: In this model, disclosed data $U$ is designed which maximizes the information released about $Y$ and satisfies the sparse point-wise privacy leakage constraint. Here, each realization of $U$ must satisfy two constraints: 1. each $u$ can be correlated with at most $N$ letters of $X$; and 2. the total leakage from $X$ to $u$ is bounded. Here, $\mathcal{L}(X \rightarrow u_i)$ denotes the amount of leakage from $X$ (through the correlated letters) to $u \in \mathcal{U}$.
• Figure 2: Tightness of the SDP relaxation as a function of the leakage budget $N$. Beyond a sparsity threshold $N_{\mathrm{th}}^{(\mathrm{emp})}$, the convex SDP relaxation (with simple rounding) matches the exact sparse Pareto optimum.

Theorems & Definitions (23)

• Remark 1
• Remark 2
• Remark 3
• Example 1
• Example 2
• Remark 4
• Lemma 1
• proof
• Proposition 1
• proof