Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information Density Bounds for Privacy

Sara Saeidian, Leonhard Grosse, Parastoo Sadeghi, Mikael Skoglund, Tobias J. Oechtering

TL;DR

The paper addresses how privacy can be guaranteed by enforcing a lower bound on information density between private data $X$ and public data $Y$. It introduces Pointwise Maximal Cost (PMC) as an operational, risk-averse leakage measure and shows that an upper bound on PMC is equivalent to a lower bound on the information density $i_{P_{XY}}(X;Y)$, with PMC equivalently characterized by $D_ ty(P_X ullmid P_{X|Y=y})$ in both randomized-function and cost-function formulations. It establishes tight connections between PMC, PML, LIP, ALIP, and LDP in both finite and general alphabets, showing PMC can be strictly more stringent and, in general, more DP-aligned than PML. The work provides concrete PMC computations for canonical privacy mechanisms (generalized randomized response, Laplace, Gaussian) and proves key theorems (e.g., PMC$ ightleftharpoons$PML relationships and LDP implications) that guide mechanism design and the selection of privacy frameworks. Overall, the results bridge gaps between privacy definitions, enabling principled comparisons and design choices across DP-like and information-density-based privacy paradigms, with clear implications for utility-privacy tradeoffs in real-world applications.

Abstract

This paper explores the implications of guaranteeing privacy by imposing a lower bound on the information density between the private and the public data. We introduce an operationally meaningful privacy measure called pointwise maximal cost (PMC) and demonstrate that imposing an upper bound on PMC is equivalent to enforcing a lower bound on the information density. PMC quantifies the information leakage about a secret to adversaries who aim to minimize non-negative cost functions after observing the outcome of a privacy mechanism. When restricted to finite alphabets, PMC can equivalently be defined as the information leakage to adversaries aiming to minimize the probability of incorrectly guessing randomized functions of the secret. We study the properties of PMC and apply it to standard privacy mechanisms to demonstrate its practical relevance. Through a detailed examination, we connect PMC with other privacy measures that impose upper or lower bounds on the information density. Our results highlight that lower bounding the information density is a more stringent requirement than upper bounding it. Overall, our work significantly bridges the gaps in understanding the relationships between various privacy frameworks and provides insights for selecting a suitable framework for a given application.

Information Density Bounds for Privacy

TL;DR

The paper addresses how privacy can be guaranteed by enforcing a lower bound on information density between private data and public data . It introduces Pointwise Maximal Cost (PMC) as an operational, risk-averse leakage measure and shows that an upper bound on PMC is equivalent to a lower bound on the information density , with PMC equivalently characterized by in both randomized-function and cost-function formulations. It establishes tight connections between PMC, PML, LIP, ALIP, and LDP in both finite and general alphabets, showing PMC can be strictly more stringent and, in general, more DP-aligned than PML. The work provides concrete PMC computations for canonical privacy mechanisms (generalized randomized response, Laplace, Gaussian) and proves key theorems (e.g., PMCPML relationships and LDP implications) that guide mechanism design and the selection of privacy frameworks. Overall, the results bridge gaps between privacy definitions, enabling principled comparisons and design choices across DP-like and information-density-based privacy paradigms, with clear implications for utility-privacy tradeoffs in real-world applications.

Abstract

This paper explores the implications of guaranteeing privacy by imposing a lower bound on the information density between the private and the public data. We introduce an operationally meaningful privacy measure called pointwise maximal cost (PMC) and demonstrate that imposing an upper bound on PMC is equivalent to enforcing a lower bound on the information density. PMC quantifies the information leakage about a secret to adversaries who aim to minimize non-negative cost functions after observing the outcome of a privacy mechanism. When restricted to finite alphabets, PMC can equivalently be defined as the information leakage to adversaries aiming to minimize the probability of incorrectly guessing randomized functions of the secret. We study the properties of PMC and apply it to standard privacy mechanisms to demonstrate its practical relevance. Through a detailed examination, we connect PMC with other privacy measures that impose upper or lower bounds on the information density. Our results highlight that lower bounding the information density is a more stringent requirement than upper bounding it. Overall, our work significantly bridges the gaps in understanding the relationships between various privacy frameworks and provides insights for selecting a suitable framework for a given application.
Paper Structure (31 sections, 16 theorems, 131 equations, 1 figure)

This paper contains 31 sections, 16 theorems, 131 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Motivations
  3. Contributions and Summary of the Main Results
  4. Other Related Works
  5. Preliminaries
  6. Notation and Assumptions
  7. Pointwise Maximal Leakage
  8. Local Information Privacy and Asymmetric Local Information Privacy
  9. Local Differential Privacy
  10. Quantifying Information Leaked to Risk-averse Adversaries
  11. Randomized Function Model
  12. Cost Function Model
  13. Relationship to Maximal Cost Leakage and Maximal Realizable Cost
  14. Relationship to Maximal Guesswork Leakage
  15. PMC on General Alphabets
  16. ...and 16 more sections

Key Result

Proposition 1

Let $X$ and $Y$ be finite random variables and suppose $X$ is distributed according to $P_X$. Given $\varepsilon \geq 0$, define If $P_{Y \mid X}$ satisfies $\varepsilon$-LDP, then it also satisfies $\varepsilon_1$-LIP, $(\varepsilon_1, \varepsilon_2)$-ALIP and $\varepsilon_2$-PML.

Figures (1)

  • Figure 1: Relationship between the upper and lower bounds on the information density. The blue curve corresponds to $\varepsilon^*_l (\varepsilon_u)$ and the orange curve corresponds to $\varepsilon^*_u (\varepsilon_l)$. When $P_X$ is binary and uniform, the two curves coincide.

Theorems & Definitions (40)

  • Definition 1: Rényi divergence of order $\infty$ van2014renyi
  • Definition 2: Gain function view of PML saeidian2023pointwisegeneral
  • Definition 3: Randomized function view of PML saeidian2023pointwise
  • Definition 4: Local information privacy jiang2018context6483382
  • Definition 5: Asymmetric local information privacy zarrabian2023lift
  • Definition 6: Local differential privacy
  • Remark 1
  • Proposition 1: jiang2021LIPcontextaware
  • Definition 7: Randomized function view of PMC
  • Theorem 1
  • ...and 30 more