Robust ML Auditing using Prior Knowledge

TL;DR

The paper tackles audit manipulation by platforms during fairness evaluations and proposes a manipulation-proof auditing framework that leverages the auditor's prior knowledge of the task. It formalizes the concept of an auditor prior, including a dataset prior D_a, and analyzes the conditions under which an auditor can detect manipulated models h_m against an honest baseline h_p. It shows that public priors can be exploited, while private dataset priors yield measurable detection guarantees, deriving a closed-form expression for the detection probability P_{uf} in terms of the prior distance δ and the risk threshold τ. Through experiments on tabular (ACSEmployment) and vision (CelebA) tasks, the work demonstrates that platforms can conceal substantial unfairness (10–20 DP points) under feasible budgets, though larger audit budgets reduce concealment in some settings. These results offer a principled path toward more robust, prior-informed fairness audits and point to future work on continuous, adaptive auditing mechanisms to sustain accountability.

Abstract

Among the many technical challenges to enforcing AI regulations, one crucial yet underexplored problem is the risk of audit manipulation. This manipulation occurs when a platform deliberately alters its answers to a regulator to pass an audit without modifying its answers to other users. In this paper, we introduce a novel approach to manipulation-proof auditing by taking into account the auditor's prior knowledge of the task solved by the platform. We first demonstrate that regulators must not rely on public priors (e.g. a public dataset), as platforms could easily fool the auditor in such cases. We then formally establish the conditions under which an auditor can prevent audit manipulations using prior knowledge about the ground truth. Finally, our experiments with two standard datasets illustrate the maximum level of unfairness a platform can hide before being detected as malicious. Our formalization and generalization of manipulation-proof auditing with a prior opens up new research directions for more robust fairness audits.

Figures (4)

• Figure 1: The auditing process as conducted by an auditor, which proceeds in three steps. The platform exposes a model $h_p$ to the users. To appear fair to the auditor while not deteriorating the utility for its users, the platform manipulates its answers on the audit set $S$.
• Figure 2: Representation of the auditor prior $\mathcal{H}_a$, the honest platform model $h_p$ and a corresponding malicious model $h_m$ on the fair $\mathcal{F}$ plane. The red area represents the area where platforms optimal manipulations are detected as dishonest: they fall outside of the blue region of $\mathcal{F}$
• Figure 3: The concealable unfairness by the platform for different detection scores and manipulation strategies. We highlight this for two features of the CelebA dataset (left) and for two different ML models trained on the ACSEmployment dataset (right). The horizontal red line indicates the DP of the most unfair model without manipulation.
• Figure 4: The concealable unfairness for different audit budgets (, data samples from the labeled dataset). We highlight this for two features of the CelebA dataset (left) and for two different ML models trained on the ACSEmployment dataset (right).

Theorems & Definitions (22)

• Definition 3.1: Auditor prior
• Theorem 3.2
• proof
• Definition 4.1: Dataset prior
• Definition 4.2: Detection rate
• Theorem 4.3: Prior-Uniform detection rate
• Corollary 4.3: Detection rate lower bound
• Lemma 1.1
• proof
• Definition 1.2