An Information Geometric Approach to Local Information Privacy with Applications to Max-lift and Local Differential Privacy

TL;DR

This work addresses designing privacy mechanisms under Local Information Privacy (LIP) to maximize information about useful data $Y$ while protecting private data $X$. It uses information-geometric local approximations to linearize the mutual information $I(U;Y)$ in the small-leakage regime, transforming the problem into a quadratic form involving the top singular value of a matrix $W$ that encodes the leakage structure. Two complementary approaches are proposed: (i) a lower-bound method based on bounding $\log(1+x)$ and perturbation constraints, yielding $P_1$, and (ii) a direct quadratic-approximation method yielding $P_2$, with $P_2\ge P_1$; both lead to low-complexity privacy designs that depend on principal directions (singular vectors) of $W$. The framework is generalizable to max-lift leakage and Local Differential Privacy (LDP), and is validated numerically, showing competitive performance and scalable design in the high-privacy regime.

Abstract

We study an information-theoretic privacy mechanism design, where an agent observes useful data $Y$ and wants to reveal the information to a user. Since the useful data is correlated with the private data $X$, the agent uses a privacy mechanism to produce disclosed data $U$ that can be released. We assume that the agent observes $Y$ and has no direct access to $X$, i.e., the private data is hidden. We study the privacy mechanism design that maximizes the revealed information about $Y$ while satisfying a bounded Local Information Privacy (LIP) criterion. When the leakage is sufficiently small, concepts from information geometry allow us to locally approximate the mutual information. By utilizing this approximation the main privacy-utility trade-off problem can be rewritten as a quadratic optimization problem that has closed-form solution under some constraints. For the cases where the closed-form solution is not obtained we provide lower bounds on it. In contrast to the previous works that have complexity issues, here, we provide simple privacy designs with low complexity which are based on finding the maximum singular value and singular vector of a matrix. To do so, we follow two approaches where in the first one we find a lower bound on the main problem and then approximate it, however, in the second approach we approximate the main problem directly. In this work, we present geometrical interpretations of the proposed methods and in a numerical example we compare our results considering both approaches with the optimal solution and the previous methods. Furthermore, we discuss how our method can be generalized considering larger amounts for the privacy leakage. Finally, we discuss how the proposed methods can be applied to deal with differential privacy.

Figures (5)

• Figure 1: In this model, disclosed data $U$ is designed by a privacy mechanism that maximizes the information disclosed about $Y$ and satisfies the bounded LIP criterion. Here, we assume that the private data $X$ is not available directly to the agent.
• Figure 2: Feasible sets in the first approach. Here, $S_1$ and $S_2$ correspond to the points satisfying \ref{['c1']} and \ref{['jadid']}, respectively. The circle corresponds to the unit $\ell_2$-ball.
• Figure 3: Feasible sets in the second approach. Here, $S_1$ and $S_2$ correspond to the points satisfying \ref{['c1']} and \ref{['c3']}, respectively. The circle corresponds to the unit $\ell_2$-ball. Compared to the first approach, the feasible set is larger which leads to higher utilities.
• Figure 4: Acheivability of Lemma \ref{['lem2']} and Lemma \ref{['lem3']}.
• Figure 5: Comparing the proposed methods in this paper with the optimal solution and previous method in khodam. It can be seen that in the high privacy regimes, $P_2$ is close to the exact solution which is found by exhaustive search and $P_2$ dominates $P_1$.

Theorems & Definitions (44)

• Remark 1
• Remark 2
• Proposition 1
• proof
• Proposition 2
• Lemma 1
• proof
• Corollary 1
• proof
• Remark 3