Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OnlineMate: An LLM-Based Multi-Agent Companion System for Cognitive Support in Online Learning

Xian Gao, Zongyun Zhang, Ting Liu, Yuzhuo Fu

TL;DR

OnlineMate addresses the lack of personalized peer interaction in online learning by integrating Theory of Mind (ToM) into a multi-agent LLM framework. The system uses a Classroom Context Manager, a Classroom Behavior Controller, and an Evaluation Agent to infer learners’ cognitive and emotional states and provide cognitive scaffolding tailored to Bloom’s Taxonomy levels. Across simulation, human, and real-classroom trials, OnlineMate increases average cognitive levels by roughly one Bloom level and enhances emotional engagement, with ablation studies clarifying the contributions of ToM reasoning and scaffolding. The results indicate substantial potential for scalable, personalized cognitive support in online learning, while also outlining limitations related to generalizability, long-term efficacy, and system scalability.

Abstract

In online learning environments, students often lack personalized peer interactions, which are crucial for cognitive development and learning engagement. Although previous studies have employed large language models (LLMs) to simulate interactive learning environments, these interactions are limited to conversational exchanges, failing to adapt to learners' individualized cognitive and psychological states. As a result, students' engagement is low and they struggle to gain inspiration. To address this challenge, we propose OnlineMate, a multi-agent learning companion system driven by LLMs integrated with Theory of Mind (ToM). OnlineMate simulates peer-like roles, infers learners' psychological states such as misunderstandings and confusion during collaborative discussions, and dynamically adjusts interaction strategies to support higher-order thinking. Comprehensive evaluations, including simulation-based experiments, human assessments, and real classroom trials, demonstrate that OnlineMate significantly promotes deep learning and cognitive engagement by elevating students' average cognitive level while substantially improving emotional engagement scores.

OnlineMate: An LLM-Based Multi-Agent Companion System for Cognitive Support in Online Learning

TL;DR

OnlineMate addresses the lack of personalized peer interaction in online learning by integrating Theory of Mind (ToM) into a multi-agent LLM framework. The system uses a Classroom Context Manager, a Classroom Behavior Controller, and an Evaluation Agent to infer learners’ cognitive and emotional states and provide cognitive scaffolding tailored to Bloom’s Taxonomy levels. Across simulation, human, and real-classroom trials, OnlineMate increases average cognitive levels by roughly one Bloom level and enhances emotional engagement, with ablation studies clarifying the contributions of ToM reasoning and scaffolding. The results indicate substantial potential for scalable, personalized cognitive support in online learning, while also outlining limitations related to generalizability, long-term efficacy, and system scalability.

Abstract

In online learning environments, students often lack personalized peer interactions, which are crucial for cognitive development and learning engagement. Although previous studies have employed large language models (LLMs) to simulate interactive learning environments, these interactions are limited to conversational exchanges, failing to adapt to learners' individualized cognitive and psychological states. As a result, students' engagement is low and they struggle to gain inspiration. To address this challenge, we propose OnlineMate, a multi-agent learning companion system driven by LLMs integrated with Theory of Mind (ToM). OnlineMate simulates peer-like roles, infers learners' psychological states such as misunderstandings and confusion during collaborative discussions, and dynamically adjusts interaction strategies to support higher-order thinking. Comprehensive evaluations, including simulation-based experiments, human assessments, and real classroom trials, demonstrate that OnlineMate significantly promotes deep learning and cognitive engagement by elevating students' average cognitive level while substantially improving emotional engagement scores.

Paper Structure

This paper contains 44 sections, 11 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLMs in Education
  4. LLMs for Human Simulation
  5. Theory of Mind (ToM) in LLMs
  6. OnlineMate Framework
  7. Classroom Context Manager
  8. Classroom Behavior Controller
  9. Evaluation Agent
  10. Role and scenario construction
  11. Multi-turn dialogue simulation
  12. Emotional and cognitive evaluation
  13. OnlineMate Agent
  14. ToM Hypothesis Generation
  15. Hypothesis Refinement and Filtering
  16. ...and 29 more sections

Figures (11)

  • Figure 1: The composition of OnlineMate framework and its workflow.
  • Figure 2: The workflow of the Evaluation Agent.
  • Figure 3: The complete workflow of OnlineMate Agents from obtaining input to generating responses.
  • Figure 4: The average cognitive level exhibited in student utterances facilitated by OnlineMate measured at varying dialogue turns.
  • Figure 5: The impact of varying the number of OnlineMate Agents on the highest cognitive level attained by students.
  • ...and 6 more figures