MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring

TL;DR

MMTutorBench introduces the first multimodal benchmark explicitly designed for AI math tutoring, evaluating 12 MLLMs on 685 problems structured around pedagogically significant key-steps. The dataset pairs handwritten-student frames with three tutoring tasks—Insight Discovery, Operation Formulation, and Operation Execution—assessed via a six-dimension rubric and a rubric-guided LLM judge. Findings reveal a clear gap between proprietary and open-source models, and a persistent gap from human tutors, with end-to-end visual grounding proving essential and OCR-based pipelines significantly underperforming. The study highlights the diagnostic value of rubric-based evaluation for tutoring and sets a foundation for advancing safe, effective AI math tutors, while noting limitations such as single-turn evaluation and English-language scope.

Abstract

Effective math tutoring requires not only solving problems but also diagnosing students' difficulties and guiding them step by step. While multimodal large language models (MLLMs) show promise, existing benchmarks largely overlook these tutoring skills. We introduce MMTutorBench, the first benchmark for AI math tutoring, consisting of 685 problems built around pedagogically significant key-steps. Each problem is paired with problem-specific rubrics that enable fine-grained evaluation across six dimensions, and structured into three tasks-Insight Discovery, Operation Formulation, and Operation Execution. We evaluate 12 leading MLLMs and find clear performance gaps between proprietary and open-source systems, substantial room compared to human tutors, and consistent trends across input variants: OCR pipelines degrade tutoring quality, few-shot prompting yields limited gains, and our rubric-based LLM-as-a-Judge proves highly reliable. These results highlight both the difficulty and diagnostic value of MMTutorBench for advancing AI tutoring.

Figures (8)

• Figure 1: (a) Existing benchmarks usually target a single perspective, such as handwritten expression recognition or problem solving, which is insufficient for evaluating tutoring ability in real educational settings. (b) An example from MMTutorBench: we model the tutoring process in realistic classroom scenarios by taking a student’s handwritten solution attempt and help-seeking question as input. The tutoring response is structured along three dimensions: Insight, Formulation, and Execution. We emphasize some key guidance in bold for illustration.
• Figure 2: The data curation pipeline of MMTutorBench. We start by collecting problems including both images and questions. The model are instructed to fulfill 3 tutoring tasks for the input problem. The generated response are evaluated with carefully designed problem specific rubrics.
• Figure 3: Performance comparison of various models across five distinct input variants. The variants include: (a) Zero-Shot, where only the images is provided; (b) With Query, which supplements the images with a corresponding textual student query; (c) OCR-Text, a pipeline approach where text is first extracted from the images via an OCR model and then fed to the language model; (d) 1-Shot and (e) 3-Shot, which provide one and three in-context examples, respectively. The results highlight the significant performance boost from including student queries and the critical limitations of the OCR-based pipeline approach.
• Figure 4: Error distribution for the top-performing model, Gemini-2.5-Pro. The chart displays the percentage of samples that received a score of zero in each of our six evaluation dimensions.
• Figure 5: Inter-judge reliability of our evaluation rubric. The plot shows the correlation between average scores assigned to all 12 MLLMs by two independent judge models: GPT-o4-mini and Qwen3-30B-A3B-Instruct-2507.