Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Auditing for Racial Discrimination in the Delivery of Education Ads

Basileal Imana, Aleksandra Korolova, John Heidemann

TL;DR

This study investigates whether Meta's ad-delivery algorithms discriminate by race in the delivery of education advertisements. It introduces a paired-ad methodology that uses for-profit and public colleges with known de-facto enrollment skews to isolate algorithmic effects from confounding factors, leveraging DMA- and race-based audience inferences. Across neutral and realistic ad creatives, the authors find evidence of racial bias in ad delivery, with for-profit college ads reaching a higher fraction of Black users, and effects amplified when using realistic imagery and in schools previously scrutinized legally. The findings underscore the need for broader, independent auditing of ad-delivery systems across domains, clarifying potential legal liabilities and urging regulators and platforms to adopt standardized, transparent scrutiny mechanisms.

Abstract

Digital ads on social-media platforms play an important role in shaping access to economic opportunities. Our work proposes and implements a new third-party auditing method that can evaluate racial bias in the delivery of ads for education opportunities. Third-party auditing is important because it allows external parties to demonstrate presence or absence of bias in social-media algorithms. Education is a domain with legal protections against discrimination and concerns of racial-targeting, but bias induced by ad delivery algorithms has not been previously explored in this domain. Prior audits demonstrated discrimination in platforms' delivery of ads to users for housing and employment ads. These audit findings supported legal action that prompted Meta to change their ad-delivery algorithms to reduce bias, but only in the domains of housing, employment, and credit. In this work, we propose a new methodology that allows us to measure racial discrimination in a platform's ad delivery algorithms for education ads. We apply our method to Meta using ads for real schools and observe the results of delivery. We find evidence of racial discrimination in Meta's algorithmic delivery of ads for education opportunities, posing legal and ethical concerns. Our results extend evidence of algorithmic discrimination to the education domain, showing that current bias mitigation mechanisms are narrow in scope, and suggesting a broader role for third-party auditing of social media in areas where ensuring non-discrimination is important.

Auditing for Racial Discrimination in the Delivery of Education Ads

TL;DR

This study investigates whether Meta's ad-delivery algorithms discriminate by race in the delivery of education advertisements. It introduces a paired-ad methodology that uses for-profit and public colleges with known de-facto enrollment skews to isolate algorithmic effects from confounding factors, leveraging DMA- and race-based audience inferences. Across neutral and realistic ad creatives, the authors find evidence of racial bias in ad delivery, with for-profit college ads reaching a higher fraction of Black users, and effects amplified when using realistic imagery and in schools previously scrutinized legally. The findings underscore the need for broader, independent auditing of ad-delivery systems across domains, clarifying potential legal liabilities and urging regulators and platforms to adopt standardized, transparent scrutiny mechanisms.

Abstract

Digital ads on social-media platforms play an important role in shaping access to economic opportunities. Our work proposes and implements a new third-party auditing method that can evaluate racial bias in the delivery of ads for education opportunities. Third-party auditing is important because it allows external parties to demonstrate presence or absence of bias in social-media algorithms. Education is a domain with legal protections against discrimination and concerns of racial-targeting, but bias induced by ad delivery algorithms has not been previously explored in this domain. Prior audits demonstrated discrimination in platforms' delivery of ads to users for housing and employment ads. These audit findings supported legal action that prompted Meta to change their ad-delivery algorithms to reduce bias, but only in the domains of housing, employment, and credit. In this work, we propose a new methodology that allows us to measure racial discrimination in a platform's ad delivery algorithms for education ads. We apply our method to Meta using ads for real schools and observe the results of delivery. We find evidence of racial discrimination in Meta's algorithmic delivery of ads for education opportunities, posing legal and ethical concerns. Our results extend evidence of algorithmic discrimination to the education domain, showing that current bias mitigation mechanisms are narrow in scope, and suggesting a broader role for third-party auditing of social media in areas where ensuring non-discrimination is important.
Paper Structure (20 sections, 2 equations, 7 figures, 4 tables)

This paper contains 20 sections, 2 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology: Using Pair of Schools with De-Facto Racial Skew in Enrollment
  3. Identifying a De-facto Skew In Education
  4. Identifying a Pair of Education Opportunities to Minimize Confounding Factors
  5. Running Ads on Meta and Evaluating their Delivery
  6. Building Audiences so that Location Reports Correspond to Race
  7. Selecting Ad Creatives and Campaign Parameters
  8. Launching and Monitoring Performance
  9. Evaluating Skew
  10. Experiments and Results
  11. Demonstrating Discriminatory Ad Delivery using Neutral Ad Creatives
  12. Demonstrating Amplified Skew Using Realistic Ad Creatives
  13. Experiments Using Schools with Prior Legal Scrutiny
  14. Discussion
  15. Platforms Shaping Access to Education Opportunities
  16. ...and 5 more sections

Figures (7)

  • Figure 1: List of pairs of schools we use in our experiments on delivery of education ads. For each school, the table shows the racial makeup of the student body ("B" = Black students, "W" = White students, "O" = Other) and the admission rate.
  • Figure 2: Example ad creatives for studying racial skew in delivery of education ads. The two left figures use neutral ad creatives that do not include people. The two right figures use realistic creatives taken from each school's ad library page and include people of a specific perceived race.
  • Figure 3: Results for Meta's delivery of educations ads for neutral creatives (left) and realistic creatives (right). Bars show 95% confidence intervals around each fraction. $n$ is the number of individuals each ad was shown to. $D$ is the difference between fraction of Blacks seeing for-profit and public school ads. $Z$ is the test statistic for significance of this difference. An audience named "aud-nc-*" is built using Black individuals from DMA group 1 (Table \ref{['tab:voter_data_list']}) and White individuals from group 2; "aud-nc-*f" is a flipped version.
  • Figure 4: Statistical significance of racial skew in delivery of education ads on Meta. The test statistic is computed based on the racial skew measured in Figure \ref{['fig:racial_skew_results']}. The racial skew in delivery between a pair of ads is statistically significant if the test statistic bar is above the horizontal line (which corresponds to a 95% confidence level: $Z_{\alpha} = 1.64$). Each bar corresponds to an experiment.
  • Figure 5: Skew measured by comparing the delivery of ads for private schools with prior legal scrutiny and public schools.
  • ...and 2 more figures