Limitations of Pinned AUC for Measuring Unintended Bias

TL;DR

This paper critiques the Pinned AUC bias metric, showing it can misrepresent unintended bias when identity subgroups do not share identical label distributions. It provides a Mann-Whitney U–based decomposition to explain why unequal sample sizes across terms distort the metric and demonstrates this with experiments on toxicity classifiers and a skewed synthetic test set. The results reveal that Pinned AUC can mask real bias under realistic data conditions, motivating the use of fates-2019 threshold-agnostic metrics that are robust to distribution differences and offer a richer bias analysis. The work advocates adopting these robust metrics as a general framework for measuring unintended bias.

Abstract

This report examines the Pinned AUC metric introduced and highlights some of its limitations. Pinned AUC provides a threshold-agnostic measure of unintended bias in a classification model, inspired by the ROC-AUC metric. However, as we highlight in this report, there are ways that the metric can obscure different kinds of unintended biases when the underlying class distributions on which bias is being measured are not carefully controlled.