From Motion Signals to Insights: A Unified Framework for Student Behavior Analysis and Feedback in Physical Education Classes

TL;DR

Addresses the challenge of analyzing student behavior in physical education by replacing video with IMU motion signals and introduces a unified end-to-end framework that cascades Motion Detection, Activity Recognition, and Action Quality Assessment to produce structured results that are fed to a large language model for reporting. The method is pre-trained on large public and synthetic datasets to generalize across diverse PE activities and is fine-tuned with few real-world samples using LoRA, with windowing parameters $L_w$ and $L_s$ guiding motion segmentation. Experimental results show strong motion-detection metrics around 0.92, high activity-recognition accuracy after fine-tuning, and action-quality scoring that correlates well with human assessments, outperforming video-based baselines. Case studies demonstrate automated individual and class reports that reflect pedagogical insights and instructional design recommendations, illustrating practical impact for teachers and students.

Abstract

Analyzing student behavior in educational scenarios is crucial for enhancing teaching quality and student engagement. Existing AI-based models often rely on classroom video footage to identify and analyze student behavior. While these video-based methods can partially capture and analyze student actions, they struggle to accurately track each student's actions in physical education classes, which take place in outdoor, open spaces with diverse activities, and are challenging to generalize to the specialized technical movements involved in these settings. Furthermore, current methods typically lack the ability to integrate specialized pedagogical knowledge, limiting their ability to provide in-depth insights into student behavior and offer feedback for optimizing instructional design. To address these limitations, we propose a unified end-to-end framework that leverages human activity recognition technologies based on motion signals, combined with advanced large language models, to conduct more detailed analyses and feedback of student behavior in physical education classes. Our framework begins with the teacher's instructional designs and the motion signals from students during physical education sessions, ultimately generating automated reports with teaching insights and suggestions for improving both learning and class instructions. This solution provides a motion signal-based approach for analyzing student behavior and optimizing instructional design tailored to physical education classes. Experimental results demonstrate that our framework can accurately identify student behaviors and produce meaningful pedagogical insights.

Figures (5)

• Figure 1: The overall architecture of our proposed framework. The cascaded motion detection, activity recognition, and action quality assessment models process students' IMU motion signals into structured tuple data (visualized as a timeline) readable by large language models. The large language model then generates analytical reports as feedback for both students and teachers.
• Figure 2: The architecture and corresponding processing workflows of MD, AR, and AQA model.
• Figure 3: Heatmap of the confusion matrices.
• Figure 4: An excerpt from a personal analysis report generated by LLM. The highlighted sections indicate content that human evaluators identified as pedagogically significant.
• Figure 5: An excerpt from a class analysis report generated by LLM.