Auditing Work: Exploring the New York City algorithmic bias audit regime

TL;DR

NYC's Local Law 144 initiates the first government-backed, independent bias audit regime for AEDTs in hiring, but the study finds it underdeveloped, with vague definitions, undefined auditor roles, and limited public audits hindering its effectiveness for jobseekers. Through 16 expert interviews and grounded-theory analysis, the authors map practical auditing components, the relational dynamics among auditors, employers, and vendors, and identify four distinct auditor roles. They reveal substantial disagreements about who qualifies as a legitimate auditor, data-access hurdles, and a reliance on a flawed theory of change that leans on transparency rather than enforceable remediation. The paper offers four policy recommendations—clear definitions and metrics, standardized auditor oversight, smoother data access, and an ecosystem approach with central repositories—to make audits a meaningful accountability mechanism in high-stakes AI use. The findings carry practical implications for policymakers crafting similar regimes at national or regional levels and contribute to the broader discourse on algorithmic accountability in employment contexts.

Abstract

In July 2023, New York City (NYC) initiated the first algorithm auditing system for commercial machine-learning systems. Local Law 144 (LL 144) mandates NYC-based employers using automated employment decision-making tools (AEDTs) in hiring to undergo annual bias audits conducted by an independent auditor. This paper examines lessons from LL 144 for other national algorithm auditing attempts. Through qualitative interviews with 16 experts and practitioners within the regime, we find that LL 144 has not effectively established an auditing regime. The law fails to clearly define key aspects, such as AEDTs and independent auditors, leading auditors, AEDT vendors, and companies using AEDTs to define the law's practical implementation in ways that failed to protect job applicants. Contributing factors include the law's flawed transparency-driven theory of change, industry lobbying narrowing the definition of AEDTs, practical and cultural challenges faced by auditors in accessing data, and wide disagreement over what constitutes a legitimate auditor, resulting in four distinct 'auditor roles.' We conclude with four recommendations for policymakers seeking to create similar bias auditing regimes, emphasizing clearer definitions, metrics, and increased accountability. By exploring LL 144 through the lens of auditors, our paper advances the evidence base around audit as an accountability mechanism, providing guidance for policymakers seeking to create similar regimes.

Figures (4)

• Figure 1: History of LL 144
• Figure 2: Services offered under the LL44 regime by auditors
• Figure 3: Example audit report, prepared by ConductorAI for NBC
• Figure : Table 2: Participant ID