Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models

Zheng Li, Siyao Song, Jingyuan Ma, Rui Li, Ying Zeng, Minghao Li, Zhifang Sui

TL;DR

This work treats teaching ability as a distinct dimension of LLM behavior and introduces a syllabus-grounded framework that evaluates teaching by measuring student performance gains after multi-turn instruction, all while preventing information leakage through structured knowledge points. It builds a knowledge-structure tree from Gaokao syllabi, uses LLMs to tag questions and generate practice problems, and employs web-enabled retrieval to source instructional material. Experiments on Gaokao data reveal that teaching effectiveness varies across subjects and models, with math, history, and politics easier to teach than physics and chemistry, and show that example problems do not universally improve teaching due to a shift toward error-correction behavior. Overall, the paper establishes that teaching ability is measurable and domain-dependent, offering a foundation for future research in AI-assisted pedagogy and teacher-agent design.

Abstract

Large language models (LLMs) show promise as teaching assistants, yet their teaching capability remains insufficiently evaluated. Existing benchmarks mainly focus on problem-solving or problem-level guidance, leaving knowledge-centered teaching underexplored. We propose a syllabus-grounded evaluation framework that measures LLM teaching capability via student performance improvement after multi-turn instruction. By restricting teacher agents to structured knowledge points and example problems, the framework avoids information leakage and enables reuse of existing benchmarks. We instantiate the framework on Gaokao data across multiple subjects. Experiments reveal substantial variation in teaching effectiveness across models and domains: some models perform well in mathematics, while teaching remains challenging in physics and chemistry. We also find that incorporating example problems does not necessarily improve teaching, as models often shift toward example-specific error correction. Overall, our results highlight teaching ability as a distinct and measurable dimension of LLM behavior.

TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models

TL;DR

This work treats teaching ability as a distinct dimension of LLM behavior and introduces a syllabus-grounded framework that evaluates teaching by measuring student performance gains after multi-turn instruction, all while preventing information leakage through structured knowledge points. It builds a knowledge-structure tree from Gaokao syllabi, uses LLMs to tag questions and generate practice problems, and employs web-enabled retrieval to source instructional material. Experiments on Gaokao data reveal that teaching effectiveness varies across subjects and models, with math, history, and politics easier to teach than physics and chemistry, and show that example problems do not universally improve teaching due to a shift toward error-correction behavior. Overall, the paper establishes that teaching ability is measurable and domain-dependent, offering a foundation for future research in AI-assisted pedagogy and teacher-agent design.

Abstract

Large language models (LLMs) show promise as teaching assistants, yet their teaching capability remains insufficiently evaluated. Existing benchmarks mainly focus on problem-solving or problem-level guidance, leaving knowledge-centered teaching underexplored. We propose a syllabus-grounded evaluation framework that measures LLM teaching capability via student performance improvement after multi-turn instruction. By restricting teacher agents to structured knowledge points and example problems, the framework avoids information leakage and enables reuse of existing benchmarks. We instantiate the framework on Gaokao data across multiple subjects. Experiments reveal substantial variation in teaching effectiveness across models and domains: some models perform well in mathematics, while teaching remains challenging in physics and chemistry. We also find that incorporating example problems does not necessarily improve teaching, as models often shift toward example-specific error correction. Overall, our results highlight teaching ability as a distinct and measurable dimension of LLM behavior.
Paper Structure (25 sections, 2 figures, 4 tables)

This paper contains 25 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Answer-centric benchmarks for knowledge and reasoning.
  4. Real-exam and Chinese-context evaluation.
  5. Tutoring datasets and pedagogy-oriented benchmarks.
  6. Method
  7. Knowledge Structure Tree
  8. LLM-based Question Tagger
  9. LLM-based Question Generator
  10. Model choice and web-enabled retrieval.
  11. Difficulty calibration and output specification.
  12. Post-generation validation and self-consistency checks.
  13. Teacher Agent
  14. Student Agent
  15. Teaching Procedure
  16. ...and 10 more sections

Figures (2)

  • Figure 1: Overview of the proposed LLM teaching evaluation framework. The workflow consists of three stages: (1) Knowledge Structuring, which transforms the raw syllabus and questions into a structured knowledge tree with associated exercises; (2) Teaching Loop, which simulates multi-turn, dialogue-based interactions and instruction between the target LLM acting as the teacher agent and a student agent; and (3) Evaluation, which quantifies student performance improvement by comparing results before and after the interaction, yielding a final teaching ability score.
  • Figure 2: Average number of dialogue turns required by different models to complete a teaching session.