Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Testing for racial bias using inconsistent perceptions of race

Nora Gera, Emma Pierson

TL;DR

The paper addresses measuring racial bias by exploiting within-person variation in perceived race across encounters, avoiding cross-person confounds. It formalizes a fixed-effects framework, modeling $y_{it} = _i +  X_{it} +  r_{it} + _{it}$ to estimate how a change in perceived race affects treatment, with $$ capturing the bias effect. Applied to Open Policing data from Arizona, Colorado, and Texas, the study finds that the same driver is 0.4 percentage points more likely to be searched when perceived as Hispanic than white, with robustness across controls and alternative models, and similar results for arrest rates. The approach generalizes to other domains where identity attributes are perceived rather than self-reported, and where the same individuals are observed repeatedly, offering a new tool for detecting bias in healthcare, surveys, criminal justice, and child welfare contexts.

Abstract

Tests for racial bias commonly assess whether two people of different races are treated differently. A fundamental challenge is that, because two people may differ in many ways, factors besides race might explain differences in treatment. Here, we propose a test for bias which circumvents the difficulty of comparing two people by instead assessing whether the $\textit{same person}$ is treated differently when their race is perceived differently. We apply our method to test for bias in police traffic stops, finding that the same driver is likelier to be searched or arrested by police when they are perceived as Hispanic than when they are perceived as white. Our test is broadly applicable to other datasets where race, gender, or other identity data are perceived rather than self-reported, and the same person is observed multiple times.

Testing for racial bias using inconsistent perceptions of race

TL;DR

The paper addresses measuring racial bias by exploiting within-person variation in perceived race across encounters, avoiding cross-person confounds. It formalizes a fixed-effects framework, modeling to estimate how a change in perceived race affects treatment, with capturing the bias effect. Applied to Open Policing data from Arizona, Colorado, and Texas, the study finds that the same driver is 0.4 percentage points more likely to be searched when perceived as Hispanic than white, with robustness across controls and alternative models, and similar results for arrest rates. The approach generalizes to other domains where identity attributes are perceived rather than self-reported, and where the same individuals are observed repeatedly, offering a new tool for detecting bias in healthcare, surveys, criminal justice, and child welfare contexts.

Abstract

Tests for racial bias commonly assess whether two people of different races are treated differently. A fundamental challenge is that, because two people may differ in many ways, factors besides race might explain differences in treatment. Here, we propose a test for bias which circumvents the difficulty of comparing two people by instead assessing whether the is treated differently when their race is perceived differently. We apply our method to test for bias in police traffic stops, finding that the same driver is likelier to be searched or arrested by police when they are perceived as Hispanic than when they are perceived as white. Our test is broadly applicable to other datasets where race, gender, or other identity data are perceived rather than self-reported, and the same person is observed multiple times.
Paper Structure (9 sections, 1 equation, 4 figures, 1 table)

This paper contains 9 sections, 1 equation, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Results
  3. Motivation
  4. Empirical implementation
  5. Policing data
  6. Estimates of bias
  7. Discussion
  8. Data processing
  9. Supplementary Figures

Figures (4)

  • Figure 1: Estimated increase in the search rate when the same driver is perceived as Hispanic as opposed to white, using a linear probability model. All estimates include driver fixed effects. 95% confidence intervals are plotted with standard errors clustered at the driver level. Estimates remain similar when including controls for officer identity, stop location, and stop date and time.
  • Figure S1: Estimated increase in the arrest rate (as opposed to the search rate, as in our primary specification) when the same driver is perceived as Hispanic as opposed to white, using a linear probability model. All estimates include driver fixed effects. 95% confidence intervals are plotted with standard errors clustered at the driver level. Estimates use Colorado and Arizona data because Texas does not provide arrest data. The finding of bias against Hispanic drivers remains robust when using this alternate outcome, and including controls for officer, stop location, and stop date/time.
  • Figure S2: Estimates from a fixed effects generalized linear model with a logit link. The horizontal axis plots the coefficient on driver race = Hispanic after controlling for driver fixed effects. The finding of bias against Hispanic drivers remains robust when using this alternate statistical model, and including controls for officer, stop location, and stop date/time.
  • Figure S3: Estimates from a conditional logistic regression model breslow1978estimation with a stratum for each driver. The horizontal axis plots the coefficient on driver race = Hispanic. The finding of bias against Hispanic drivers remains robust when using this alternate statistical model, and including controls for officer, stop location, and stop date/time.