MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Education

TL;DR

MathVC tackles the challenge of sustaining CPS in middle school mathematics by introducing a multi-persona LLM-based virtual classroom. It deploys a modular system with meta planning and a persona simulation stack, including task schemas and error-injected personas, to orchestrate CPS dialogues. An evaluation with 14 middle school students shows improvements in engagement, motivation, and mathematical confidence, along with nuanced insights into authenticity and socio-emotional dynamics. The work provides design guidance for future AI-enabled collaborative learning tools and discusses ethical considerations and generalizability beyond the middle school context.

Abstract

Collaborative problem solving (CPS) is essential in mathematics education, fostering deeper learning through the exchange of ideas. Yet, classrooms often lack the resources, time, and peer dynamics needed to sustain productive CPS. Recent advancements in Large Language Models (LLMs) offer a promising avenue to enhance CPS in mathematical education. We designed and developed MathVC, a multi-persona LLM simulated virtual classroom platform to facilitate CPS in mathematics. MathVC combines a meta planning controller that monitors CPS stages-sense-making, team organization, planning, execution, validation, and predicts the next speaker, with a persona simulation stack that encodes mathematical thinking via a task schema and error-injected persona schemas seeded from teacher-specified misconceptions. We evaluated MathVC with 14 U.S. middle schoolers. Students reported constructive interaction and reaching shared solutions, describing gains in engagement, motivation, and confidence through diverse perspectives, immediate scaffolding, and human-like fallibility. Our findings also provide insights into simulating peers via LLM-based technologies for collaboration to support learning.

Figures (10)

• Figure 1: MathVC user interface has several components: (1) Question Control and Display, enabling students to select and navigate among mathematical problems; (2) a Chat Interface, where the human student interacts with multiple LLM-simulated virtual peers; and (3) the User Input Box, allowing students to send messages and engage collaboratively with virtual peers. In this example session as shown, a middle-school student (anonymized) and two virtual peers (Alice and Bob) collaboratively discuss and solve a mathematical problem.
• Figure 2: Overview of MathVC's structure. (1) Initialization: The administrator (middle school teacher) first sets up a mathematics problem and a set of virtual peers (e.g., Alice in the example). The task schema generator then automatically generates a structured representation of the problem solution to facilitate the CPS. (2) Persona Simulation: When a user (middle school student) starts a session on MathVC, the specified virtual peers are created, where the persona schema generator generates a structured representation of each persona's understanding of the problem by potentially altering correct variable values (e.g., changing 15 to 5 and 2 to 6) to reflect the persona's mathematical skill level (e.g., Alice's the limited skill). During CPS, the persona schema will also be updated by persona schema modifier to reflect the latest knowledge of the persona. Subsequently, the dialogue act generator and the response generator decide the intent and the content of each virtual peer's next utterance, which is then sent to the frontend for display. (3) Meta Planning: The entire CPS dialogue is overseen by the collaborative stage monitor, which decides the current stage of CPS, and the next speaker control, which selects the next speaker (e.g., Alice).
• Figure 3: Example dialogues among three LLM-simulated virtual students (Alice, Bob, and Charlie) in our preliminary exploration. We manually annotated the simulated stages in blue and indicated responses reflecting Alice's limited math skill level in red. Vanilla simulation by prompting an LLM with the persona specification leads to lengthy responses and unfaithful simulation---for example, Alice merely verbally indicates that she struggles with math but still gives a correct calculation "48". In addition, vanilla simulation often results in very short conversations, where virtual peers start directly from solution sharing and end immediately after each of them describes their solution, limiting the engagement of human students and their effective math learning. In contrast, MathVC enables much more realistic simulation, where Alice makes a genuine mistake during problem solving, asks for an explanation from her peers, and performs self-reflection to evolve her understanding. In our simulation, the virtual peers also speak in much shorter utterances, closer to how middle school students communicate with each other. The overall conversation becomes much more extended with natural transitions from the team establishing shared understanding, dividing workload, to collaborative problem-solving and answer verification, which thus allows for more effective participation when deployed to serve human students.
• Figure 4: Prompt template for the next speaker control module.
• Figure 5: Prompt template for collaborative stage monitor.