Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Developing Authentic Simulated Learners for Mathematics Teacher Learning: Insights from Three Approaches with Large Language Models

Jie Cao, Ha Nguyen, Selim Yavuz, Boran Yu, Shuguang Wang, Pavneet Kaur Bharaj, Dionne Cross Francis

Abstract

Large Language Model (LLM) simulations, where LLMs act as students with varying approaches to learning tasks, can support teachers' noticing of student thinking. However, simulations using zero- or few-shot prompting often yield inauthentic knowledge and language, directing teachers to unrealistic reasoning. We evaluate three approaches (Fine-tuning, Multi-agent, and Direct Preference Optimization; DPO) to improve the authenticity and pedagogical utility of simulated students. All approaches improve cognitive and linguistic authenticity, compared with few-shot prompts. Interviews with elementary mathematics pre-service teachers and researchers (\textit{n} = 8) reveal distinct pedagogical affordances. The fine-tuned model produces realistic, brief responses but limits opportunities to extend students' thinking. Meanwhile, the multi-agent and DPO approaches generate explicit reasoning behind student strategies. We discuss implications for designing LLM simulations that balance authenticity with instructional utility for teacher learning.

Developing Authentic Simulated Learners for Mathematics Teacher Learning: Insights from Three Approaches with Large Language Models

Abstract

Large Language Model (LLM) simulations, where LLMs act as students with varying approaches to learning tasks, can support teachers' noticing of student thinking. However, simulations using zero- or few-shot prompting often yield inauthentic knowledge and language, directing teachers to unrealistic reasoning. We evaluate three approaches (Fine-tuning, Multi-agent, and Direct Preference Optimization; DPO) to improve the authenticity and pedagogical utility of simulated students. All approaches improve cognitive and linguistic authenticity, compared with few-shot prompts. Interviews with elementary mathematics pre-service teachers and researchers (\textit{n} = 8) reveal distinct pedagogical affordances. The fine-tuned model produces realistic, brief responses but limits opportunities to extend students' thinking. Meanwhile, the multi-agent and DPO approaches generate explicit reasoning behind student strategies. We discuss implications for designing LLM simulations that balance authenticity with instructional utility for teacher learning.

Paper Structure

This paper contains 18 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. Teacher Noticing in Mathematics Learning
  4. LLM-based Student Simulations
  5. Method
  6. The Teaching Task and Simulated Student Profile
  7. Evaluation Framework of the Simulated Student
  8. Three Approaches for Developing the Simulated Student
  9. Multi-agent
  10. Fine-tuning
  11. DPO
  12. Research Procedures
  13. RQ1: Authenticity of the Approaches, Compared to Baseline Prompts
  14. RQ2: Teachers' Feedback on Authenticity and Pedagogical Utility
  15. Results
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Prompt and interface for the simulated agent (Josh)
  • Figure 2: Overview of the approaches. LLM selection considering commercial (e.g., GPT) and open-source LLMs (Llama, Mistral) from testing-feedback in Summer-Fall 2025.
  • Figure 3: Overall performance of the three approaches: Cognition and Language
  • Figure 4: Participants' rankings for preference and authenticity