On Information Theoretic Fairness With A Bounded Point-Wise Statistical Parity Constraint: An Information Geometric Approach

TL;DR

This work addresses designing fair representations under a bounded point-wise parity constraint by leveraging information-geometric approximations to reduce the problem to a tractable quadratic form. For small fairness thresholds, the complex utility–fairness–compression trade-off is captured via perturbations around the distributions of the sensitive attribute and task, enabling a simple, constructive design based on dominant singular vectors of two key matrices. The main contributions are a provable quadratic approximation $P_2$, singular-value-based lower bounds, and low-complexity designs that closely approach optimal performance in high-privacy regimes, as demonstrated on a numerical example. The framework offers a practical method for fairness-aware data representation with controlled leakage, with potential extensions to broader per-letter and sparsity-based constraints in information-theoretic privacy and fairness.

Abstract

In this paper, we study an information-theoretic problem of designing a fair representation under a bounded point-wise statistical (demographic) parity constraint. More specifically, an agent uses some useful data (database) $X$ to solve a task $T$. Since both $X$ and $T$ are correlated with some latent sensitive attribute or secret $S$, the agent designs a representation $Y$ that satisfies a bounded point-wise statistical parity, that is, such that for all realizations of the representation $y\in\cal Y$, we have $χ^2(P_{S|y};P_S)\leq ε$. In contrast to our previous work, here we use the point-wise measure instead of a bounded mutual information, and we assume that the agent has no direct access to $S$ and $T$; hence, the Markov chains $S - X - Y$ and $T - X - Y$ hold. In this work, we design $Y$ that maximizes the mutual information $I(Y;T)$ about the task while satisfying a bounded compression rate constraint, that is, ensuring that $I(Y;X) \leq r$. Finally, $Y$ satisfies the point-wise bounded statistical parity constraint $χ^2(P_{S|y};P_S)\leq ε$. When $ε$ is small, concepts from information geometry allow us to locally approximate the KL-divergence and mutual information. To design the representation $Y$, we utilize this approximation and show that the main complex fairness design problem can be rewritten as a quadratic optimization problem that has simple closed-form solution under certain constraints. For the cases where the closed-form solution is not obtained we obtain lower bounds with low computational complexity. Here, we provide simple fairness designs with low complexity which are based on finding the maximum singular value and singular vector of a matrix. Finally, in a numerical example we compare our obtained results with the optimal solution.

Figures (2)

• Figure 1: Fair mechanism design under bounded point-wise statistical parity. The goal is to design $Y$ using database $X$ that is useful for the task $T$, is compressed, and leaks within a controlled threshold of the latent sensitive attribute $S$. Here, the agent has no direct access to $S$ and $T$.
• Figure 2: Comparing the proposed approach in this paper with the optimal solutions of \ref{['problem']} and \ref{['prob2']}. We can see that in the high privacy regimes, $P_2$ is close to the exact solution of \ref{['problem']} which is found by exhaustive search and $g^{r}_{\epsilon}(P_{S,X,T})$ dominates $g^{r}_{\epsilon,\chi^2}(P_{S,X,T})$.

Theorems & Definitions (14)

• Remark 1
• Remark 2
• Remark 3
• Lemma 1
• proof
• Theorem 1
• Remark 4
• Remark 5
• Proposition 1
• proof