Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Using Large Language Models to Detect Socially Shared Regulation of Collaborative Learning

Jiayi Zhang, Conrad Borchers, Clayton Cohn, Namrata Srivastava, Caitlin Snyder, Siyuan Guo, Ashwin T S, Naveeduddin Mohammed, Haley Noh, Gautam Biswas

TL;DR

This study addresses detecting socially shared regulation of learning (SSRL) in collaborative STEM+C tasks by leveraging multimodal data (discourse + environment logs) and large language models (LLMs). It treats LLMs as controlled summarizers to produce task-aware segment representations and compares text-only, context-enriched, log-derived, and multimodal features for SSRL prediction using nested cross-validated neural models. Results show text-only embeddings often yield the strongest SSRL detection performance, with context and log features providing targeted benefits for planning and reflection, respectively; multimodal inputs offer competitive gains when environment traces align with behaviors. The findings support scalable SSRL detection to enable real-time feedback and adaptive scaffolding in collaborative learning, while incorporating teacher perspectives on usability and interpretability. This work advances learning analytics by integrating discourse with system logs in a principled, evaluative framework and highlights directions for richer multimodal features and broader applicability across domains.

Abstract

The field of learning analytics has made notable strides in automating the detection of complex learning processes in multimodal data. However, most advancements have focused on individualized problem-solving instead of collaborative, open-ended problem-solving, which may offer both affordances (richer data) and challenges (low cohesion) to behavioral prediction. Here, we extend predictive models to automatically detect socially shared regulation of learning (SSRL) behaviors in collaborative computational modeling environments using embedding-based approaches. We leverage large language models (LLMs) as summarization tools to generate task-aware representations of student dialogue aligned with system logs. These summaries, combined with text-only embeddings, context-enriched embeddings, and log-derived features, were used to train predictive models. Results show that text-only embeddings often achieve stronger performance in detecting SSRL behaviors related to enactment or group dynamics (e.g., off-task behavior or requesting assistance). In contrast, contextual and multimodal features provide complementary benefits for constructs such as planning and reflection. Overall, our findings highlight the promise of embedding-based models for extending learning analytics by enabling scalable detection of SSRL behaviors, ultimately supporting real-time feedback and adaptive scaffolding in collaborative learning environments that teachers value.

Using Large Language Models to Detect Socially Shared Regulation of Collaborative Learning

TL;DR

This study addresses detecting socially shared regulation of learning (SSRL) in collaborative STEM+C tasks by leveraging multimodal data (discourse + environment logs) and large language models (LLMs). It treats LLMs as controlled summarizers to produce task-aware segment representations and compares text-only, context-enriched, log-derived, and multimodal features for SSRL prediction using nested cross-validated neural models. Results show text-only embeddings often yield the strongest SSRL detection performance, with context and log features providing targeted benefits for planning and reflection, respectively; multimodal inputs offer competitive gains when environment traces align with behaviors. The findings support scalable SSRL detection to enable real-time feedback and adaptive scaffolding in collaborative learning, while incorporating teacher perspectives on usability and interpretability. This work advances learning analytics by integrating discourse with system logs in a principled, evaluative framework and highlights directions for richer multimodal features and broader applicability across domains.

Abstract

The field of learning analytics has made notable strides in automating the detection of complex learning processes in multimodal data. However, most advancements have focused on individualized problem-solving instead of collaborative, open-ended problem-solving, which may offer both affordances (richer data) and challenges (low cohesion) to behavioral prediction. Here, we extend predictive models to automatically detect socially shared regulation of learning (SSRL) behaviors in collaborative computational modeling environments using embedding-based approaches. We leverage large language models (LLMs) as summarization tools to generate task-aware representations of student dialogue aligned with system logs. These summaries, combined with text-only embeddings, context-enriched embeddings, and log-derived features, were used to train predictive models. Results show that text-only embeddings often achieve stronger performance in detecting SSRL behaviors related to enactment or group dynamics (e.g., off-task behavior or requesting assistance). In contrast, contextual and multimodal features provide complementary benefits for constructs such as planning and reflection. Overall, our findings highlight the promise of embedding-based models for extending learning analytics by enabling scalable detection of SSRL behaviors, ultimately supporting real-time feedback and adaptive scaffolding in collaborative learning environments that teachers value.
Paper Structure (16 sections, 2 figures, 2 tables)

This paper contains 16 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Self-Regulated Learning and Socially Shared Regulation of Learning
  4. Large Language Models for Self-Regulated Learning Measurements
  5. Methods
  6. Learning Environment and Study Design
  7. Data Processing and SSRL Coding
  8. Predictive Modeling of SSRL Categories
  9. Data Preparation
  10. Model Training and Evaluation
  11. Results
  12. Discussion
  13. Main Findings
  14. Implications and Teacher Feedback
  15. Limitations and Future Work
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Example solution for the C2STEM Truck Task with Task Context categories cohn2025personalizing.
  • Figure 2: Multimodal data processing pipeline showing audio (blue) and logs (yellow) processing, and how the data was fused (green). Red boxes indicate data used for analysis in this paper (Merged Actions/Discourse as inputs and SRL Labels as outputs), which is represented by the dashed line and discussed shortly. All icons are c/o Flaticon.