Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reducing the Filtering Effect in Public School Admissions: A Bias-aware Analysis for Targeted Interventions

Yuri Faenza, Swati Gupta, Aapeli Vuorinen, Xuan Zhang

TL;DR

This paper analyzes a bias problem in New York City SHSAT-based public high school admissions, where SES-related disparities create a large filtering effect in access to top schools. It introduces a continuous two-group matching model with a multiplicative bias $\beta$ on disadvantaged perceived potential and analyzes both deterministic and randomized pre-admission interventions (vouchers) to debias the system, providing explicit policies and theoretical guarantees. The authors derive optimal deterministic debiasing ranges, prove their limitations for incentive compatibility and individual fairness, and then propose randomized policies, notably Proportional-to-Mistreatment (PropM), that are incentive compatible and Lipschitz, often achieving lower maximum mistreatment than any deterministic policy. The experimental validation on the 2016-17 NYC SHSAT data shows that average-performing disadvantaged students experience the most mistreatment under bias and that targeted randomized vouchers can substantially reduce mistreatment while maintaining fairness, offering actionable guidance for resource allocation in education policy with robust results under various assumptions about student preferences and potential distributions.

Abstract

Problem definition: Traditionally, New York City's top 8 public schools have selected candidates solely based on their scores in the Specialized High School Admissions Test (SHSAT). These scores are known to be impacted by socioeconomic status of students and test preparation received in middle schools, leading to a massive filtering effect in the education pipeline. The classical mechanisms for assigning students to schools do not naturally address problems like school segregation and class diversity, which have worsened over the years. The scientific community, including policymakers, have reacted by incorporating group-specific quotas and proportionality constraints, with mixed results. The problem of finding effective and fair methods for broadening access to top-notch education is still unsolved. Methodology/results: We take an operations approach to the problem different from most established literature, with the goal of increasing opportunities for students with high economic needs. Using data from the Department of Education (DOE) in New York City, we show that there is a shift in the distribution of scores obtained by students that the DOE classifies as "disadvantaged" (following criteria mostly based on economic factors). We model this shift as a "bias" that results from an underestimation of the true potential of disadvantaged students. We analyze the impact this bias has on an assortative matching market. We show that centrally planned interventions can significantly reduce the impact of bias through scholarships or training, when they target the segment of disadvantaged students with average performance.

Reducing the Filtering Effect in Public School Admissions: A Bias-aware Analysis for Targeted Interventions

TL;DR

This paper analyzes a bias problem in New York City SHSAT-based public high school admissions, where SES-related disparities create a large filtering effect in access to top schools. It introduces a continuous two-group matching model with a multiplicative bias on disadvantaged perceived potential and analyzes both deterministic and randomized pre-admission interventions (vouchers) to debias the system, providing explicit policies and theoretical guarantees. The authors derive optimal deterministic debiasing ranges, prove their limitations for incentive compatibility and individual fairness, and then propose randomized policies, notably Proportional-to-Mistreatment (PropM), that are incentive compatible and Lipschitz, often achieving lower maximum mistreatment than any deterministic policy. The experimental validation on the 2016-17 NYC SHSAT data shows that average-performing disadvantaged students experience the most mistreatment under bias and that targeted randomized vouchers can substantially reduce mistreatment while maintaining fairness, offering actionable guidance for resource allocation in education policy with robust results under various assumptions about student preferences and potential distributions.

Abstract

Problem definition: Traditionally, New York City's top 8 public schools have selected candidates solely based on their scores in the Specialized High School Admissions Test (SHSAT). These scores are known to be impacted by socioeconomic status of students and test preparation received in middle schools, leading to a massive filtering effect in the education pipeline. The classical mechanisms for assigning students to schools do not naturally address problems like school segregation and class diversity, which have worsened over the years. The scientific community, including policymakers, have reacted by incorporating group-specific quotas and proportionality constraints, with mixed results. The problem of finding effective and fair methods for broadening access to top-notch education is still unsolved. Methodology/results: We take an operations approach to the problem different from most established literature, with the goal of increasing opportunities for students with high economic needs. Using data from the Department of Education (DOE) in New York City, we show that there is a shift in the distribution of scores obtained by students that the DOE classifies as "disadvantaged" (following criteria mostly based on economic factors). We model this shift as a "bias" that results from an underestimation of the true potential of disadvantaged students. We analyze the impact this bias has on an assortative matching market. We show that centrally planned interventions can significantly reduce the impact of bias through scholarships or training, when they target the segment of disadvantaged students with average performance.

Paper Structure

This paper contains 30 sections, 18 theorems, 61 equations, 10 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. A continuous matching market
  5. Continuous Model with One Group:
  6. Continuous Model with Two Groups:
  7. Impact on Students
  8. Deterministic Centralized Interventions
  9. Metric of Impact on Students:
  10. Fairness Considerations:
  11. Optimal Deterministic Strategies:
  12. Incentive Compatible and Individually Fair Voucher Distribution
  13. Randomized assignment of vouchers
  14. Discussion on technical assumptions.
  15. Experimental Case Study
  16. ...and 15 more sections

Key Result

Proposition 1

For any student $\theta \in G_2$, the displacement $\widehat{\mu}(\theta) - \mu(\theta)$ is given by: For any student $\theta\in G_1$, we have $\widehat{\mu}(\theta)-\mu(\theta) = \left( -p+p\beta^\alpha \right) \left({Z(\theta)} \right)^{-\alpha}.$ Thus, the maximum displacement of $(1-p)(1-\beta^\alpha)$ is experienced by a $G_2$ student with potential $1/\beta$; and the most significant negati

Figures (10)

  • Figure 1: Distribution of SHSAT scores across students in group $G_1$ versus the distribution of SHSAT scores across students in group $G_2$ for the 2016-17 academic year. We estimate the bias factor $\beta \approx 0.88$.
  • Figure 2: Distribution of true potentials (scaled SHSAT scores) of students who score high enough to receive an offer from a SHS. The best fitting Pareto distribution (i.e., the "theoretical pdf" curve) has parameter $\alpha=8.9$.
  • Figure 3: Schools students "should" versus "actually" attend. The green dotted line is a line of slope one, representing the place a student should be placed if there is no bias in the system.
  • Figure 4: Effect of bias after debiasing the optimal set of $G_2$ students given $\widehat{c}$.
  • Figure 5: Maximum mistreatment before and after optimal voucher correction.
  • ...and 5 more figures

Theorems & Definitions (22)

  • Example 1
  • Example 2
  • Proposition 1
  • Theorem 1
  • Theorem 2
  • Lemma 1
  • Theorem 3
  • Lemma 2
  • Theorem 4
  • Proposition EC.1
  • ...and 12 more