Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Do Test Scores Help Teachers Give Better Track Advice to Students? A Principal Stratification Analysis

Andrea Ichino, Fabrizia Mealli, Javier Viviens

Abstract

Every year, over one million EU students choose a secondary school track based on teacher recommendations, yet little evidence shows this yields optimal assignments. Using Dutch data, we examine whether access to standardized test scores improves recommendation quality. We develop a Principal-Stratification metric in a quasi-randomized setting, conduct a welfare analysis that flexibly weights short- and long-term losses, and assess principal fairness by examining whether test-score access affects equity across protected attributes. Results are robust to replacing the Exclusion Restriction assumption underlying our main identification strategy with alternative assumptions. Allowing recommendation upgrades when test scores exceed expectations increases successful placement in more demanding tracks by at least 6%, while misplacing 7% of weaker students. Only unrealistically high weights on short-term losses would justify banning such upgrades. Test-score access also yields fairer recommendations for immigrant and low-SES students. Our methodology and findings contribute to the literature on algorithm-assisted human decisions.

Do Test Scores Help Teachers Give Better Track Advice to Students? A Principal Stratification Analysis

Abstract

Every year, over one million EU students choose a secondary school track based on teacher recommendations, yet little evidence shows this yields optimal assignments. Using Dutch data, we examine whether access to standardized test scores improves recommendation quality. We develop a Principal-Stratification metric in a quasi-randomized setting, conduct a welfare analysis that flexibly weights short- and long-term losses, and assess principal fairness by examining whether test-score access affects equity across protected attributes. Results are robust to replacing the Exclusion Restriction assumption underlying our main identification strategy with alternative assumptions. Allowing recommendation upgrades when test scores exceed expectations increases successful placement in more demanding tracks by at least 6%, while misplacing 7% of weaker students. Only unrealistically high weights on short-term losses would justify banning such upgrades. Test-score access also yields fairer recommendations for immigrant and low-SES students. Our methodology and findings contribute to the literature on algorithm-assisted human decisions.

Paper Structure

This paper contains 13 sections, 2 theorems, 84 equations, 6 figures, 17 tables.

Table of Contents

  1. Introduction
  2. The Dutch school system and the quasi-experiment
  3. Statistical framework
  4. Assumptions
  5. Students' strata
  6. The metric to compare recommendations
  7. Identification of the $APCE_J$
  8. Test scores and quality of teachers' advice: evidence
  9. Estimates for the three strata
  10. A comprehensive evaluation
  11. An alternative to assuming the Exclusion Restriction
  12. Fairness of teachers' recommendations
  13. Conclusions

Key Result

Theorem 10

Point identification of APCEs under unconfoundedness: Under assumptions 1, a:rand_sourc, a:ec, a:strata_monoton, and a:unconf, $APCE_{AL}$, $APCE_{AH}$ and $APCE_{H}$ are identified as follows: where Proof: See Imai2023.

Figures (6)

  • Figure 1: Time-line for track recommendations and outcomes
  • Figure 2: Comprehensive evaluation that test score information improves teachers' recommendations
  • Figure 3: Overview of the $APCE_J$ estimates
  • Figure 4: Fairness for immigrants
  • Figure 5: Fairness by SES
  • ...and 1 more figures

Theorems & Definitions (3)

  • Definition 9
  • Theorem 10
  • Theorem 11