Benchmarking Educational LLMs with Analytics: A Case Study on Gender Bias in Feedback
Yishan Du, Conrad Borchers, Mutlu Cukurova
TL;DR
Motivated by persistent gender bias in educational LLM feedback, this paper proposes an embedding-based benchmarking framework using counterfactual gender cues in AES-2.0 essays. It computes semantic drift between original and counterfactual feedback via cosine distance $d_{cos}$ and Euclidean distance $d_{euclid}$ across six models and assesses significance with a permutation test. Results show asymmetric responses to gender substitutions: implicit cues induce consistent bias across models, while explicit cues yield model-dependent effects; qualitative analysis reveals linguistic and pedagogical biases that could shape learner agency and competence. The work provides a practical fairness-auditing methodology for educational GenAI and informs prompt design and deployment to safeguard learner equity.
Abstract
As teachers increasingly turn to GenAI in their educational practice, we need robust methods to benchmark large language models (LLMs) for pedagogical purposes. This article presents an embedding-based benchmarking framework to detect bias in LLMs in the context of formative feedback. Using 600 authentic student essays from the AES 2.0 corpus, we constructed controlled counterfactuals along two dimensions: (i) implicit cues via lexicon-based swaps of gendered terms within essays, and (ii) explicit cues via gendered author background in the prompt. We investigated six representative LLMs (i.e. GPT-5 mini, GPT-4o mini, DeepSeek-R1, DeepSeek-R1-Qwen, Gemini 2.5 Pro, Llama-3-8B). We first quantified the response divergence with cosine and Euclidean distances over sentence embeddings, then assessed significance via permutation tests, and finally, visualised structure using dimensionality reduction. In all models, implicit manipulations reliably induced larger semantic shifts for male-female counterfactuals than for female-male. Only the GPT and Llama models showed sensitivity to explicit gender cues. These findings show that even state-of-the-art LLMs exhibit asymmetric semantic responses to gender substitutions, suggesting persistent gender biases in feedback they provide learners. Qualitative analyses further revealed consistent linguistic differences (e.g., more autonomy-supportive feedback under male cues vs. more controlling feedback under female cues). We discuss implications for fairness auditing of pedagogical GenAI, propose reporting standards for counterfactual evaluation in learning analytics, and outline practical guidance for prompt design and deployment to safeguard equitable feedback.