Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Student Classroom Behavior Detection based on Spatio-Temporal Network and Multi-Model Fusion

Fan Yang, Xiaofei Wang

TL;DR

This work proposed a method for extending the spatio-temporal behavior dataset in Student Classroom Scenarios (SCB-ST-Dataset4) through image dataset, and proposed a Behavior Similarity Index (BSI) to explore the similarity of behaviors.

Abstract

Using deep learning methods to detect students' classroom behavior automatically is a promising approach for analyzing their class performance and improving teaching effectiveness. However, the lack of publicly available spatio-temporal datasets on student behavior, as well as the high cost of manually labeling such datasets, pose significant challenges for researchers in this field. To address this issue, we proposed a method for extending the spatio-temporal behavior dataset in Student Classroom Scenarios (SCB-ST-Dataset4) through image dataset. Our SCB-ST-Dataset4 comprises 757265 images with 25810 labels, focusing on 3 behaviors: hand-raising, reading, writing. Our proposed method can rapidly generate spatio-temporal behavior datasets without requiring extra manual labeling. Furthermore, we proposed a Behavior Similarity Index (BSI) to explore the similarity of behaviors. We evaluated the dataset using the YOLOv5, YOLOv7, YOLOv8, and SlowFast algorithms, achieving a mean average precision (map) of up to 82.3%. Last, we fused multiple models to generate student behavior-related data from various perspectives. The experiment further demonstrates the effectiveness of our method. And SCB-ST-Dataset4 provides a robust foundation for future research in student behavior detection, potentially contributing to advancements in this field. The SCB-ST-Dataset4 is available for download at: https://github.com/Whiffe/SCB-dataset.

Student Classroom Behavior Detection based on Spatio-Temporal Network and Multi-Model Fusion

TL;DR

This work proposed a method for extending the spatio-temporal behavior dataset in Student Classroom Scenarios (SCB-ST-Dataset4) through image dataset, and proposed a Behavior Similarity Index (BSI) to explore the similarity of behaviors.

Abstract

Using deep learning methods to detect students' classroom behavior automatically is a promising approach for analyzing their class performance and improving teaching effectiveness. However, the lack of publicly available spatio-temporal datasets on student behavior, as well as the high cost of manually labeling such datasets, pose significant challenges for researchers in this field. To address this issue, we proposed a method for extending the spatio-temporal behavior dataset in Student Classroom Scenarios (SCB-ST-Dataset4) through image dataset. Our SCB-ST-Dataset4 comprises 757265 images with 25810 labels, focusing on 3 behaviors: hand-raising, reading, writing. Our proposed method can rapidly generate spatio-temporal behavior datasets without requiring extra manual labeling. Furthermore, we proposed a Behavior Similarity Index (BSI) to explore the similarity of behaviors. We evaluated the dataset using the YOLOv5, YOLOv7, YOLOv8, and SlowFast algorithms, achieving a mean average precision (map) of up to 82.3%. Last, we fused multiple models to generate student behavior-related data from various perspectives. The experiment further demonstrates the effectiveness of our method. And SCB-ST-Dataset4 provides a robust foundation for future research in student behavior detection, potentially contributing to advancements in this field. The SCB-ST-Dataset4 is available for download at: https://github.com/Whiffe/SCB-dataset.
Paper Structure (17 sections, 5 equations, 13 figures, 8 tables)

This paper contains 17 sections, 5 equations, 13 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. SCB-ST-Dataset4
  4. The SCB-ST-Dataset4 Production Process
  5. The SCB-ST-Dataset4 Challenge
  6. Statistics of The SCB-ST-Dataset4
  7. Spatio-Temporal Network and Multi-Model Fusion
  8. Spatio-Temporal Network
  9. Multi-Model Fusion
  10. Student Concentration Analysis
  11. Experiments
  12. Experimental Environment and Dataset
  13. Model Training
  14. Evaluation Metric
  15. Analysis of spatio-temporal detect experimental results
  16. ...and 2 more sections

Figures (13)

  • Figure 1: SlowFast detection in students' classroom.
  • Figure 2: Process for creating SCB-ST-Dataset4.
  • Figure 3: Overlapping results of detection for reading and writing behavior.
  • Figure 4: Challenges in the SCB-ST-Dataset4 include dense environments, similar behaviors, and pixel differences..
  • Figure 5: Challenges in the SCB-ST-Dataset4 include varying shooting angles, class differences, and different learning stages.
  • ...and 8 more figures