TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Models

TL;DR

TutorBench provides a rubric-driven, multimodal benchmark to evaluate how well large language models tutor students. By collecting 1490 conversations across 6 STEM subjects and 3 tutoring use cases, the benchmark uses sample-specific rubrics evaluated by an LLM-judge to yield ARR_w scores, enabling fine-grained analysis of tutoring capabilities. Evaluations of 16 frontier LLMs reveal no model surpassing 56% overall performance, with strengths and weaknesses varying by model family and use case; adaptive explanations remain particularly challenging, while active learning hints show relative strength for some models. The benchmark advances rigorous, scalable assessment of AI tutors and offers a practical resource to guide the next generation of tutoring-focused AI systems, including reproducibility provisions and a public sample dataset.

Abstract

As students increasingly adopt large language models (LLMs) as learning aids, it is crucial to build models that are adept at handling the nuances of tutoring: they need to identify the core needs of students, be adaptive, provide personalized guidance, and be accurate. To this end, we introduce TutorBench, a dataset and evaluation benchmark designed to rigorously evaluate the core tutoring skills of LLMs. The dataset comprises 1,490 samples curated by human experts, focused on high-school and AP-level curricula. The samples are drawn from three common tutoring tasks: (i) generating adaptive explanations tailored to a student's confusion, (ii) providing actionable feedback on a student's work, and (iii) promoting active learning through effective hint generation. To account for the inherent complexity of tutoring, samples are accompanied by sample-specific rubrics which are used to judge model responses during evaluation. TutorBench uses a reliable and fine-grained automatic evaluation method that uses an LLM-judge and the sample-specific rubrics. We evaluate 16 frontier LLMs on TutorBench and present a detailed analysis of their performance and behavior. Our results show that none of the frontier LLMs achieve a score of greater than $56\%$, showing a large room for improvement. We find that LLMs fall short in exhibiting the full range of tutoring skills needed to guide, diagnose, and support students effectively, with all the frontier models achieving less than a $60\%$ pass rate on rubric criteria related to these skills. We also find that different model families exhibit varied strengths and limitations: the Claude models outperform others in supporting active learning, while they lag behind in the other two use cases. By releasing TutorBench, we provide a comprehensive and unsaturated benchmark to guide the development of the next-generation of AI tutors.

Figures (12)

• Figure 1: An example from the TutorBench dataset. Each sample includes a system prompt defining the tutoring goal (top left), a question along with the student's partial work (top right), an AI model's response (bottom left), and a set of rubric criteria for evaluation (bottom right). In this instance, the AI is prompted to provide a hint for a statistics problem.
• Figure 2: Examples of two core use cases for an AI tutor. Left: The adaptive explanation generation scenario, showcasing a multi-turn dialogue in electrochemistry. The system must provide a targeted clarification in response to a student's specific follow-up question regarding standard reduction potentials. Right: The assessment and feedback scenario, where a student provides a reasoned but incorrect answer to a biology question. The system's task is to analyze the student's reasoning, identify the misconception about the Calvin cycle, and provide corrective feedback.
• Figure 3: Model performance across three use cases: Adaptive, Assessment, and Active Learning. We observe a distinct difference between the performance of the Claude family of models compared to the other models, with the Claude models performing significantly better in providing active learning support, but still lagging behind other models overall.
• Figure 4: Model performance breakdown along evaluation dimensions and Bloom's taxonomy categories. While the top-performing models GPT-5 and Gemini 2.5 Pro are close overall, their performance differs widely when measured along the above dimensions.
• Figure 5: Model performance breakdown along tutoring skills: models struggle to include alternative solutions, examples, and analogies in their responses. However, they perform relatively better in identifying mistakes, correct steps, and core misconceptions.