Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Language, Caste, and Context: Demographic Disparities in AI-Generated Explanations Across Indian and American STEM Educational Systems

Amogh Gupta, Niharika Patil, Sourojit Ghosh, SnehalKumar, S Gaikwad

TL;DR

This study tackles the problem of demographic bias in AI-generated educational explanations by conducting a cross-cultural, intersectional audit across Indian and American STEM education contexts. Using two tasks (ranking and generation) on MATH-50 and JEEBench, the authors evaluate four LLMs (two open-weight and two closed-weight) with metrics MCV, MGL, MAB, and MDB, across 100 intersectional profiles per context. They demonstrate that income is a universal predictor of more complex explanations, while context-specific patterns emerge: English-medium and higher caste in India yield more advanced outputs, and HBCU status in the U.S. aligns with simpler explanations, with biases persisting even at elite institutions. Crucially, model size or openness does not reliably mitigate bias, and biases are broadly similar across open and closed models, suggesting that such disparities are ingrained in training data. The findings underscore the need for globally inclusive fairness research, local AI tooling, and design of flexible explanations to mitigate inequitable educational support as AI becomes embedded in engineering education worldwide.

Abstract

The popularization of AI chatbot usage globally has created opportunities for research into their benefits and drawbacks, especially for students using AI assistants for coursework support. This paper asks: how do LLMs perceive the intellectual capabilities of student profiles from intersecting marginalized identities across different cultural contexts? We conduct one of the first large-scale intersectional analyses on LLM explanation quality for Indian and American undergraduate profiles preparing for engineering entrance examinations. By constructing profiles combining multiple demographic dimensions including caste, medium of instruction, and school boards in India, and race, HBCU attendance, and school type in America, alongside universal factors like income and college tier, we examine how quality varies across these factors. We observe biases providing lower-quality outputs to profiles with marginalized backgrounds in both contexts. LLMs such as Qwen2.5-32B-Instruct and GPT-4o demonstrate granular understandings of context-specific discrimination, systematically providing simpler explanations to Hindi/Regional-medium students in India and HBCU profiles in America, treating these as proxies for lower capability. Even when marginalized profiles attain social mobility by getting accepted into elite institutions, they still receive more simplistic explanations, showing how demographic information is inextricably linked to LLM biases. Different models (Qwen2.5-32B-Instruct, GPT-4o, GPT-4o-mini, GPT-OSS 20B) embed similar biases against historically marginalized populations in both contexts, preventing profiles from switching between AI assistants for better results. Our findings have strong implications for AI incorporation into global engineering education.

Language, Caste, and Context: Demographic Disparities in AI-Generated Explanations Across Indian and American STEM Educational Systems

TL;DR

This study tackles the problem of demographic bias in AI-generated educational explanations by conducting a cross-cultural, intersectional audit across Indian and American STEM education contexts. Using two tasks (ranking and generation) on MATH-50 and JEEBench, the authors evaluate four LLMs (two open-weight and two closed-weight) with metrics MCV, MGL, MAB, and MDB, across 100 intersectional profiles per context. They demonstrate that income is a universal predictor of more complex explanations, while context-specific patterns emerge: English-medium and higher caste in India yield more advanced outputs, and HBCU status in the U.S. aligns with simpler explanations, with biases persisting even at elite institutions. Crucially, model size or openness does not reliably mitigate bias, and biases are broadly similar across open and closed models, suggesting that such disparities are ingrained in training data. The findings underscore the need for globally inclusive fairness research, local AI tooling, and design of flexible explanations to mitigate inequitable educational support as AI becomes embedded in engineering education worldwide.

Abstract

The popularization of AI chatbot usage globally has created opportunities for research into their benefits and drawbacks, especially for students using AI assistants for coursework support. This paper asks: how do LLMs perceive the intellectual capabilities of student profiles from intersecting marginalized identities across different cultural contexts? We conduct one of the first large-scale intersectional analyses on LLM explanation quality for Indian and American undergraduate profiles preparing for engineering entrance examinations. By constructing profiles combining multiple demographic dimensions including caste, medium of instruction, and school boards in India, and race, HBCU attendance, and school type in America, alongside universal factors like income and college tier, we examine how quality varies across these factors. We observe biases providing lower-quality outputs to profiles with marginalized backgrounds in both contexts. LLMs such as Qwen2.5-32B-Instruct and GPT-4o demonstrate granular understandings of context-specific discrimination, systematically providing simpler explanations to Hindi/Regional-medium students in India and HBCU profiles in America, treating these as proxies for lower capability. Even when marginalized profiles attain social mobility by getting accepted into elite institutions, they still receive more simplistic explanations, showing how demographic information is inextricably linked to LLM biases. Different models (Qwen2.5-32B-Instruct, GPT-4o, GPT-4o-mini, GPT-OSS 20B) embed similar biases against historically marginalized populations in both contexts, preventing profiles from switching between AI assistants for better results. Our findings have strong implications for AI incorporation into global engineering education.
Paper Structure (38 sections, 4 equations, 7 figures, 6 tables)

This paper contains 38 sections, 4 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Algorithmic Bias and Educational Inequality
  4. LLM Bias in Educational and Social Contexts
  5. Personalization and Fairness in Predictive Modeling
  6. Building on Educational AI Bias Auditing
  7. Methodology
  8. Experimental Setup
  9. Model Configuration
  10. Student Profile Designing
  11. Datasets
  12. Experimental Tasks
  13. Metrics
  14. Results
  15. RQ1: How Do LLMs Perceive Intellectual Capabilities Across Cultural Contexts?
  16. ...and 23 more sections

Figures (7)

  • Figure 1: Experimental pipeline for measuring intersectional bias in educational AI: intersectional student profiles combining protected attributes (caste, race, income, gender, medium, college tier) from Indian and American contexts are evaluated across four LLMs through two tasks, ranking (difficulty recommendations) and generation (explanation production) with MAB and MDB metrics quantifying systematic differences in assigned instructional complexity based on demographic characteristics.
  • Figure 2: Normalized MGL ranges (lines) and means (dots) across demographic dimensions for the main model. Variability within groups is high (average span 1.4–1.9), and systematic differences between groups are evident. Many categories span nearly the full normalized range, indicating that the model produces inconsistent explanation complexity for students with identical profiles.
  • Figure 3: Maximum Difference Bias (MDB) measures the largest normalised score gap between demographic groups within each dimension, averaged across ranking and generation tasks. Across both cultural contexts, closed-source models exhibit more structured and consistently higher bias patterns, while open-source models display weaker and less uniform bias. Notably, these trends remain stable across demographic dimensions and national contexts, reinforcing the role of model design in shaping bias behaviour.
  • Figure 4: Comprehensive Statistics Across All Experiments and Models
  • Figure 6: Extreme Profiles: Highest Scores by Model and Task
  • ...and 2 more figures