Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What we learned while automating bias detection in AI hiring systems for compliance with NYC Local Law 144

Gemma Galdon Clavell, Rubén González-Sendino

TL;DR

The paper addresses how to automate bias audits for NYC Local Law 144 using ITACA_144, a lite version of ITACA_OS, to streamline compliance for AEDTs in hiring. It highlights data, metrics, and process-related challenges, proposing concrete improvements such as NYC-relevant 12-month data, elimination of problematic inclusion thresholds, and counterfactual analyses within an end-to-end audit framework. The authors emphasize lifecycle transparency, representativity-based metrics, and independent data verification to bolster audit credibility and fairness. Their work aims to provide a practical, scalable framework for regulators and practitioners, with potential applicability to other jurisdictions seeking robust bias measurement standards in AI hiring systems.

Abstract

Since July 5, 2023, New York City's Local Law 144 requires employers to conduct independent bias audits for any automated employment decision tools (AEDTs) used in hiring processes. The law outlines a minimum set of bias tests that AI developers and implementers must perform to ensure compliance. Over the past few months, we have collected and analyzed audits conducted under this law, identified best practices, and developed a software tool to streamline employer compliance. Our tool, ITACA_144, tailors our broader bias auditing framework to meet the specific requirements of Local Law 144. While automating these legal mandates, we identified several critical challenges that merit attention to ensure AI bias regulations and audit methodologies are both effective and practical. This document presents the insights gained from automating compliance with NYC Local Law 144. It aims to support other cities and states in crafting similar legislation while addressing the limitations of the NYC framework. The discussion focuses on key areas including data requirements, demographic inclusiveness, impact ratios, effective bias, metrics, and data reliability.

What we learned while automating bias detection in AI hiring systems for compliance with NYC Local Law 144

TL;DR

The paper addresses how to automate bias audits for NYC Local Law 144 using ITACA_144, a lite version of ITACA_OS, to streamline compliance for AEDTs in hiring. It highlights data, metrics, and process-related challenges, proposing concrete improvements such as NYC-relevant 12-month data, elimination of problematic inclusion thresholds, and counterfactual analyses within an end-to-end audit framework. The authors emphasize lifecycle transparency, representativity-based metrics, and independent data verification to bolster audit credibility and fairness. Their work aims to provide a practical, scalable framework for regulators and practitioners, with potential applicability to other jurisdictions seeking robust bias measurement standards in AI hiring systems.

Abstract

Since July 5, 2023, New York City's Local Law 144 requires employers to conduct independent bias audits for any automated employment decision tools (AEDTs) used in hiring processes. The law outlines a minimum set of bias tests that AI developers and implementers must perform to ensure compliance. Over the past few months, we have collected and analyzed audits conducted under this law, identified best practices, and developed a software tool to streamline employer compliance. Our tool, ITACA_144, tailors our broader bias auditing framework to meet the specific requirements of Local Law 144. While automating these legal mandates, we identified several critical challenges that merit attention to ensure AI bias regulations and audit methodologies are both effective and practical. This document presents the insights gained from automating compliance with NYC Local Law 144. It aims to support other cities and states in crafting similar legislation while addressing the limitations of the NYC framework. The discussion focuses on key areas including data requirements, demographic inclusiveness, impact ratios, effective bias, metrics, and data reliability.
Paper Structure (9 sections, 1 table)

This paper contains 9 sections, 1 table.

Table of Contents

  1. Introduction
  2. Learnings & Discussion
  3. Data requirements
  4. Demographic inclusiveness
  5. Impact ratio vs. Fairness
  6. Effective bias
  7. Metrics
  8. Data Reliability
  9. Conclussions