Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching

Yuyang Ding, Hanglei Hu, Jie Zhou, Qin Chen, Bo Jiang, Liang He

TL;DR

This work addresses the gap in math education with AI by shifting from solution-centric LLM outputs to Socratic tutoring. It proposes SocraticLLM, a knowledge-enhanced tutor that follows a four-step teaching flow (review, guidance/heuristic, rectify, summarize) and integrates extra knowledge via a prompting strategy, with efficient LoRA fine-tuning. The authors introduce SocraticMATH, a large-scale primary-school mathematics dialogue dataset with 513 knowledge points and 6,846 conversations, enriched with rich attributions for evaluation. Experimental results across automatic metrics, human judgments, and GPT-4 evaluations show that SocraticLLM yields higher reliability and better Socratic guidance than strong baselines, validating its teaching-oriented effectiveness. The work also provides code and data to enable broader adoption and further research in AI-assisted mathematical instruction.

Abstract

With the introduction of large language models (LLMs), automatic math reasoning has seen tremendous success. However, current methods primarily focus on providing solutions or using techniques like Chain-of-Thought to enhance problem-solving accuracy. In this paper, we focus on improving the capability of mathematics teaching via a Socratic teaching-based LLM (\texttt{SocraticLLM}), which guides learners toward profound thinking with clarity and self-discovery via conversation. We collect and release a high-quality mathematical teaching dataset, named \texttt{SocraticMATH}, which provides Socratic-style conversations of problems with extra knowledge. Also, we propose a knowledge-enhanced LLM as a strong baseline to generate reliable responses with review, guidance/heuristic, rectification, and summarization. Experimental results show the great advantages of \texttt{SocraticLLM} by comparing it with several strong generative models. The codes and datasets are available on \url{https://github.com/ECNU-ICALK/SocraticMath}.

Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching

TL;DR

This work addresses the gap in math education with AI by shifting from solution-centric LLM outputs to Socratic tutoring. It proposes SocraticLLM, a knowledge-enhanced tutor that follows a four-step teaching flow (review, guidance/heuristic, rectify, summarize) and integrates extra knowledge via a prompting strategy, with efficient LoRA fine-tuning. The authors introduce SocraticMATH, a large-scale primary-school mathematics dialogue dataset with 513 knowledge points and 6,846 conversations, enriched with rich attributions for evaluation. Experimental results across automatic metrics, human judgments, and GPT-4 evaluations show that SocraticLLM yields higher reliability and better Socratic guidance than strong baselines, validating its teaching-oriented effectiveness. The work also provides code and data to enable broader adoption and further research in AI-assisted mathematical instruction.

Abstract

With the introduction of large language models (LLMs), automatic math reasoning has seen tremendous success. However, current methods primarily focus on providing solutions or using techniques like Chain-of-Thought to enhance problem-solving accuracy. In this paper, we focus on improving the capability of mathematics teaching via a Socratic teaching-based LLM (\texttt{SocraticLLM}), which guides learners toward profound thinking with clarity and self-discovery via conversation. We collect and release a high-quality mathematical teaching dataset, named \texttt{SocraticMATH}, which provides Socratic-style conversations of problems with extra knowledge. Also, we propose a knowledge-enhanced LLM as a strong baseline to generate reliable responses with review, guidance/heuristic, rectification, and summarization. Experimental results show the great advantages of \texttt{SocraticLLM} by comparing it with several strong generative models. The codes and datasets are available on \url{https://github.com/ECNU-ICALK/SocraticMath}.
Paper Structure (10 sections, 2 equations, 2 figures, 3 tables)

This paper contains 10 sections, 2 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset
  4. Dataset Construction
  5. Dataset Analysis
  6. Our Method
  7. Experiments
  8. Experimental Setups.
  9. Experimental Results
  10. Conclusions and Further Work

Figures (2)

  • Figure 1: Examples of CoT and Socratic teaching.
  • Figure 2: The framework of SocraticLLM.