CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models

TL;DR

CASTLE tackles the gap in safety evaluation for student-centered AI by introducing a theory-grounded benchmark that accounts for student heterogeneity. It constructs a large-scale bilingual dataset with a 15-domain safety taxonomy and 14 attributes, enabling non-personalized vs personalized assessments across 92,908 scenarios and three targeted metrics: Risk Sensitivity, Emotional Empathy, and Student Alignment. The study demonstrates that personalization improves safety yet highlights persistent risks, with domain-specific alignment and RL-based strategies often outperform mere scaling. The work provides a foundation for context-aware safety in high-stakes educational settings and highlights practical implications for designing responsible, student-tailored AI systems.

Abstract

Large language models (LLMs) have advanced the development of personalized learning in education. However, their inherent generation mechanisms often produce homogeneous responses to identical prompts. This one-size-fits-all mechanism overlooks the substantial heterogeneity in students cognitive and psychological, thereby posing potential safety risks to vulnerable groups. Existing safety evaluations primarily rely on context-independent metrics such as factual accuracy, bias, or toxicity, which fail to capture the divergent harms that the same response might cause across different student attributes. To address this gap, we propose the concept of Student-Tailored Personalized Safety and construct CASTLE based on educational theories. This benchmark covers 15 educational safety risks and 14 student attributes, comprising 92,908 bilingual scenarios. We further design three evaluation metrics: Risk Sensitivity, measuring the model ability to detect risks; Emotional Empathy, evaluating the model capacity to recognize student states; and Student Alignment, assessing the match between model responses and student attributes. Experiments on 18 SOTA LLMs demonstrate that CASTLE poses a significant challenge: all models scored below an average safety rating of 2.3 out of 5, indicating substantial deficiencies in personalized safety assurance.

Figures (21)

• Figure 1: Student-tailored personalized safety in LLMs.Left-top (green region): Two students with different attributes ask the same query to LLMs, yet the model’s identical response leads to completely different safety outcomes—harmless for Student 1 but potentially harmful for Student 2. Left-bottom: Across more than 92k bilingual scenarios, existing models exhibit clear deficiencies in student-tailored personalized safety evaluation. Right-top (orange region): When student-specific contextual information (i.e., Student 2 attributes) is incorporated, LLMs are able to generate responses that are both safer and more empathetic. Right-bottom: This trend is consistently validated at scale, highlighting the necessity of CASTLE for evaluating personalized safety in high-risk education.
• Figure 2: Overview of our dataset construction. The upper part (Stage 0) illustrates the theoretical foundation, integrating multi-domain risk and psychological theories with logic-constraint rules. The lower part (Stages 1–3) depicts the execution flow, transitioning from real-world seed enhancement to cyclic multi-LLM profile generation and personalized query synthesis.
• Figure 3: Comparison of personalized and non-personalized safety scores across different domains and models. While incorporating student profiles significantly improves safety scores, it does not fully eliminate underlying risks.
• Figure 4: Safety score trends across different levels of profile granularity. The safety performance across all scenarios exhibits a consistent upward trend as personalized student profiles progress from missing to implicit, and finally to explicit representation.
• Figure 5: Comparison of personalized safety score across Qwen Model Scales. The impact of parameter size on safety scores is secondary to the effects of alignment and reinforcement learning in personalized scenarios.