Demographic Benchmarking: Bridging Socio-Technical Gaps in Bias Detection

TL;DR

The paper tackles the gap in AI bias assessment by moving beyond model-centric metrics to include socio-demographic, real-world population distributions. It introduces a Demographic Benchmarking Framework implemented in the ITACA auditing platform, with metrics such as Demographic Disparity $P_i - R_i$, Total Demographic Disparity $\sum_{i=1}^n |P_i - R_i|$, and Normalized Demographic Disparity $\frac{1}{n} \sum_{i=1}^n \frac{|P_i - R_i|}{P_i}$, plus positive-decision variants $DDP$, $TDDP$, and $NDDP$, to quantify disparities in both training and production. The approach enables control datasets by demographics, comparison of population vs impact population to reveal structural bias, and continuous drift monitoring across scenarios. It is demonstrated via a NYC AEDT case study under LL144, illustrating that different tools affect distinct demographic groups and underscoring regulatory gaps; overall, the framework offers a practical path for auditors, developers, and policymakers to enhance fairness and accountability in AI systems.

Abstract

Artificial intelligence (AI) models are increasingly autonomous in decision-making, making pursuing responsible AI more critical than ever. Responsible AI (RAI) is defined by its commitment to transparency, privacy, safety, inclusiveness, and fairness. But while the principles of RAI are clear and shared, RAI practices and auditing mechanisms are still incipient. A key challenge is establishing metrics and benchmarks that define performance goals aligned with RAI principles. This paper describes how the ITACA AI auditing platform developed by Eticas.ai tackles demographic benchmarking when auditing AI recommender systems. To this end, we describe a Demographic Benchmarking Framework designed to measure the populations potentially impacted by specific AI models. The framework serves us as auditors as it allows us to not just measure but establish acceptability ranges for specific performance indicators, which we share with the developers of the systems we audit so they can build balanced training datasets and measure and monitor fairness throughout the AI lifecycle. It is also a valuable resource for policymakers in drafting effective and enforceable regulations. Our approach integrates socio-demographic insights directly into AI systems, reducing bias and improving overall performance. The main contributions of this study include:1. Defining control datasets tailored to specific demographics so they can be used in model training; 2. Comparing the overall population with those impacted by the deployed model to identify discrepancies and account for structural bias; and 3. Quantifying drift in different scenarios continuously and as a post-market monitoring mechanism.

Figures (1)

• Figure 1: Flowchart illustrating the role of metrics in ensuring equal proportion against reality and usability across development and production.