TutorUp: What If Your Students Were Simulated? Training Tutors to Address Engagement Challenges in Online Learning

TL;DR

TutorUp tackles poor engagement in online tutoring by training novices with scenario-based practice using LLM-mediated simulations of students. It combines a formative two-survey study ($N_1=86$, $N_2=102$) to identify engagement challenges with a within-subject user study ($N=16$) showing TutorUp improves perceived effectiveness and tutor performance in applying engagement strategies, relative to a text-only baseline. The system features reactive disengagement scenarios, a BigPicture-Character prompting pipeline to manage dialogue, and dual feedback channels (immediate and asynchronous) grounded in learning science. The work demonstrates scalable, immersive training for online tutors and highlights future improvements in realism, scenario diversity, and measurement of engagement dynamics.

Abstract

With the rise of online learning, many novice tutors lack experience engaging students remotely. We introduce TutorUp, a Large Language Model (LLM)-based system that enables novice tutors to practice engagement strategies with simulated students through scenario-based training. Based on a formative study involving two surveys (N1=86, N2=102) on student engagement challenges, we summarize scenarios that mimic real teaching situations. To enhance immersion and realism, we employ a prompting strategy that simulates dynamic online learning dialogues. TutorUp provides immediate and asynchronous feedback by referencing tutor-students online session dialogues and evidence-based teaching strategies from learning science literature. In a within-subject evaluation (N=16), participants rated TutorUp significantly higher than a baseline system without simulation capabilities regarding effectiveness and usability. Our findings suggest that TutorUp provides novice tutors with more effective training to learn and apply teaching strategies to address online student engagement challenges.

Figures (4)

• Figure 1: TutorUp consists of multiple views and functions, including Scenario Selection (A), Scenario Description (B), Math Problem (C), and Student Information (D), which displays details for three students (D1, D2, and D3) under each scenario. The system also features an Interactive Dialogue (E), an Input Box for Tutor Instructions (F), a Get Immediate Feedback Button (G1) that provides Immediate Feedback (G2), a Reset Button (H), and a Get Asynchronous Feedback Button (J1) that provides Asynchronous Feedback (J2). Users can also click on Tutor Dialogue Bubbles To Retrieve the Instruction (I)
• Figure 2: BigPicture-Character Pipeline Demonstration: Beginning with the initial dialogue and tutor's first input, the BigPicture Agent starts to write the "story" of the dialogue and decides which Character (Tutor or Student Agent) to speak next.
• Figure 3: Baseline System: The baseline system consists of three main panels: A) Task Introduction Panel: This panel introduces the general scenario to the tutors, prompting them to consider how they would engage students in the given scenario. Tutors are then asked to write down their proposed strategies for student engagement. B) Scenario Information Panel: This panel provides the scenario's theme, the math problem, and relevant student information, all of which are also included in TutorUp. C) Feedback Panel: After tutors complete their responses, this panel offers scenario-strategy pairs as feedback. These pairs are also used as part of the feedback prompts in TutorUp.
• Figure 4: (a): Means and standard errors of baseline system and TutorUp on a 5-point Likert measured by User-Study participants; (b) Means and standard errors of baseline system and TutorUp on test results on a 3-point Likert measured by experts.