Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

P2NIA: Privacy-Preserving Non-Iterative Auditing

Jade Garcia Bourrée, Hadrien Lautraite, Sébastien Gambs, Gilles Tredan, Erwan Le Merrer, Benoît Rottembourg

TL;DR

This work tackles the problem of reliably auditing fairness in high-risk AI under black-box constraints where the platform's data distribution is hidden and privacy concerns limit API access. It introduces P2NIA, a privacy-preserving, non-iterative collaborative auditing scheme in which the platform releases a differentially-private synthetic or anonymized audit dataset to enable unbiased evaluation by the auditor. The authors show that population bias can bias black-box fairness estimates and demonstrate, on the Adult and Folktables datasets, that P2NIA achieves accurate estimates for Demographic Parity, Equalized Odds, and Equality of Opportunity under controllable $ \epsilon$-differential privacy budgets. The approach reduces auditing friction for platforms while improving audit validity, suggesting a practical path for privacy-aware governance; future work includes exploring grey-box extensions.

Abstract

The emergence of AI legislation has increased the need to assess the ethical compliance of high-risk AI systems. Traditional auditing methods rely on platforms' application programming interfaces (APIs), where responses to queries are examined through the lens of fairness requirements. However, such approaches put a significant burden on platforms, as they are forced to maintain APIs while ensuring privacy, facing the possibility of data leaks. This lack of proper collaboration between the two parties, in turn, causes a significant challenge to the auditor, who is subject to estimation bias as they are unaware of the data distribution of the platform. To address these two issues, we present P2NIA, a novel auditing scheme that proposes a mutually beneficial collaboration for both the auditor and the platform. Extensive experiments demonstrate P2NIA's effectiveness in addressing both issues. In summary, our work introduces a privacy-preserving and non-iterative audit scheme that enhances fairness assessments using synthetic or local data, avoiding the challenges associated with traditional API-based audits.

P2NIA: Privacy-Preserving Non-Iterative Auditing

TL;DR

This work tackles the problem of reliably auditing fairness in high-risk AI under black-box constraints where the platform's data distribution is hidden and privacy concerns limit API access. It introduces P2NIA, a privacy-preserving, non-iterative collaborative auditing scheme in which the platform releases a differentially-private synthetic or anonymized audit dataset to enable unbiased evaluation by the auditor. The authors show that population bias can bias black-box fairness estimates and demonstrate, on the Adult and Folktables datasets, that P2NIA achieves accurate estimates for Demographic Parity, Equalized Odds, and Equality of Opportunity under controllable -differential privacy budgets. The approach reduces auditing friction for platforms while improving audit validity, suggesting a practical path for privacy-aware governance; future work includes exploring grey-box extensions.

Abstract

The emergence of AI legislation has increased the need to assess the ethical compliance of high-risk AI systems. Traditional auditing methods rely on platforms' application programming interfaces (APIs), where responses to queries are examined through the lens of fairness requirements. However, such approaches put a significant burden on platforms, as they are forced to maintain APIs while ensuring privacy, facing the possibility of data leaks. This lack of proper collaboration between the two parties, in turn, causes a significant challenge to the auditor, who is subject to estimation bias as they are unaware of the data distribution of the platform. To address these two issues, we present P2NIA, a novel auditing scheme that proposes a mutually beneficial collaboration for both the auditor and the platform. Extensive experiments demonstrate P2NIA's effectiveness in addressing both issues. In summary, our work introduces a privacy-preserving and non-iterative audit scheme that enhances fairness assessments using synthetic or local data, avoiding the challenges associated with traditional API-based audits.

Paper Structure

This paper contains 29 sections, 2 theorems, 2 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Issues with black-box audits.
  3. Contributions.
  4. Outline.
  5. Auditing Black-Box Models
  6. Auditing Fairness Setup
  7. Fairness Definitions.
  8. Black-Box Auditing Setting.
  9. Estimation Discrepancy Due to Population Bias
  10. Auditing through a Differentially-Private Dataset
  11. P2NIA: A Non-Iterative Auditing scheme
  12. Implementing P2NIA with Differential-private Mechanisms
  13. Differential privacy.
  14. Data anonymization through local differential privacy.
  15. Mechanisms for differentially-private synthetic data generation.
  16. ...and 14 more sections

Key Result

theorem thmcountertheorem

Population bias in black-box audit. Estimating the fairness metric (e.g., demographic parity, equality of opportunity or equalized odds) of a model $m$ using a distribution $\mathcal{D}'$ at a (total variation) statistical distance $\alpha$ from the true distribution $\mathcal{D}$ can lead to an app

Figures (8)

  • Figure 1: Observed population bias on the evaluation of demographic parity with protected feature "sex" on a model trained in state CA to predict whether an individual's income is above $\$50,000$ and evaluated in other states. These results confirm the practical impact of \ref{['cor:unboundBias']}.
  • Figure 2: The P2NIA scheme steps by steps.
  • Figure 3: Demographic parity depending on the number of samples.
  • Figure 4: Demographic parity depending on the $\epsilon$-differential privacy in P2NIA.
  • Figure 5: Absolute difference compared to the reference for the three standard fairness metrics with $\epsilon = 10$, averaged on the two models.
  • ...and 3 more figures

Theorems & Definitions (3)

  • theorem thmcountertheorem
  • definition thmcounterdefinition
  • theorem thmcountertheorem