Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Analysis of Author Demographics in Academic Peer Review

Uttamasha Anjally Oyshi, Gibson Nkhata, Susan Gauch

TL;DR

Assessment of the independent impacts of author demographics, including race, gender, and country of affiliation, on paper acceptance rankings indicates statistically substantial causal disadvantages for authors from minority racial groups and those associated with institutions in the Global South.

Abstract

Academic meritocracy is jeopardized by systematic imbalances; for example, whereas Black and Hispanic individuals constitute over 30% of the U.S. population, they represent fewer than 10% of tenured academics in science and engineering. Peer review serves as a crucial gatekeeper in this process, however it encounters ongoing issues over biases that may hinder scientific advancement. The issue is now exacerbated by the growing influence of artificial intelligence (AI) in academic assessment. This paper transcends correlation to quantitatively assess the independent impacts of author demographics, including race, gender, and country of affiliation, on paper acceptance rankings. We utilize a causal inference methodology on a dataset of 530 papers, simulating the academic selection process by employing the prestige of the publication venue as a surrogate for review rank. Our research indicates statistically substantial causal disadvantages for authors from minority racial groups (average treatment effects [ATE]: -0.42 points in ranking), female authors (ATE: -0.25), and those associated with institutions in the Global South (ATE: -0.57). The exhibited biases emphasize the pressing necessity for fairness interventions in both conventional and AI-based review processes, indicating that such measures are essential for establishing a more equitable and credible scientific environment.

Causal Analysis of Author Demographics in Academic Peer Review

TL;DR

Assessment of the independent impacts of author demographics, including race, gender, and country of affiliation, on paper acceptance rankings indicates statistically substantial causal disadvantages for authors from minority racial groups and those associated with institutions in the Global South.

Abstract

Academic meritocracy is jeopardized by systematic imbalances; for example, whereas Black and Hispanic individuals constitute over 30% of the U.S. population, they represent fewer than 10% of tenured academics in science and engineering. Peer review serves as a crucial gatekeeper in this process, however it encounters ongoing issues over biases that may hinder scientific advancement. The issue is now exacerbated by the growing influence of artificial intelligence (AI) in academic assessment. This paper transcends correlation to quantitatively assess the independent impacts of author demographics, including race, gender, and country of affiliation, on paper acceptance rankings. We utilize a causal inference methodology on a dataset of 530 papers, simulating the academic selection process by employing the prestige of the publication venue as a surrogate for review rank. Our research indicates statistically substantial causal disadvantages for authors from minority racial groups (average treatment effects [ATE]: -0.42 points in ranking), female authors (ATE: -0.25), and those associated with institutions in the Global South (ATE: -0.57). The exhibited biases emphasize the pressing necessity for fairness interventions in both conventional and AI-based review processes, indicating that such measures are essential for establishing a more equitable and credible scientific environment.
Paper Structure (28 sections, 5 equations, 4 figures, 7 tables)

This paper contains 28 sections, 5 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Empirical Evidence of Bias in Academic Peer Review
  4. Causal Inference for Fairness Assessment
  5. Fairness-Aware Recommendation and Causal Interventions
  6. Methodology
  7. Problem Formalization
  8. Variable Definitions
  9. Dataset Description
  10. Statistical Estimation
  11. Propensity Score Estimation
  12. Inverse Propensity Weighting (IPW)
  13. Average Treatment Effect (ATE) Estimation
  14. Intervention and Evaluation Strategy
  15. Statistical Significance and Robustness Checks
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Causal graph illustrating how demographic attributes (Race, Gender, Country) influence paper acceptance both directly and indirectly through institutional affiliation and paper quality.
  • Figure 2: Baseline relationship between author h-index and paper acceptance probability for majority (Race=0) and minority (Race=1) authors. Minority authors have consistently lower acceptance rates than majority authors at equivalent h-index levels, indicative of bias in the review process.
  • Figure 3: Intersectional ATEs by Race and Gender across Two Estimation Strategies: IPW and Linear Regression. The minority male group is most disadvantaged in both models.
  • Figure 4: Causal Bias (ATE) vs. Fairness Regularization ($\lambda$). As $\lambda$ increases, the ATE for Race, Gender, and Country converges to zero, indicating reduced demographic bias due to fairness-aware optimization.