Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Student Classroom Behavior Detection based on YOLOv7-BRA and Multi-Model Fusion

Fan Yang, Tao Wang, Xiaofei Wang

TL;DR

The Student Classroom Behavior Detection system based on based on YOLOv7-BRA (YOLO v7 with Bi-level Routing Attention) is proposed, which identified eight different behavior patterns, including standing, sitting, speaking, listening, walking, raising hands, reading, and writing.

Abstract

Accurately detecting student behavior in classroom videos can aid in analyzing their classroom performance and improving teaching effectiveness. However, the current accuracy rate in behavior detection is low. To address this challenge, we propose the Student Classroom Behavior Detection system based on based on YOLOv7-BRA (YOLOv7 with Bi-level Routing Attention ). We identified eight different behavior patterns, including standing, sitting, speaking, listening, walking, raising hands, reading, and writing. We constructed a dataset, which contained 11,248 labels and 4,001 images, with an emphasis on the common behavior of raising hands in a classroom setting (Student Classroom Behavior dataset, SCB-Dataset). To improve detection accuracy, we added the biformer attention module to the YOLOv7 network. Finally, we fused the results from YOLOv7 CrowdHuman, SlowFast, and DeepSort models to obtain student classroom behavior data. We conducted experiments on the SCB-Dataset, and YOLOv7-BRA achieved an mAP@0.5 of 87.1%, resulting in a 2.2% improvement over previous results. Our SCB-dataset can be downloaded from: https://github.com/Whiffe/SCB-datase

Student Classroom Behavior Detection based on YOLOv7-BRA and Multi-Model Fusion

TL;DR

The Student Classroom Behavior Detection system based on based on YOLOv7-BRA (YOLO v7 with Bi-level Routing Attention) is proposed, which identified eight different behavior patterns, including standing, sitting, speaking, listening, walking, raising hands, reading, and writing.

Abstract

Accurately detecting student behavior in classroom videos can aid in analyzing their classroom performance and improving teaching effectiveness. However, the current accuracy rate in behavior detection is low. To address this challenge, we propose the Student Classroom Behavior Detection system based on based on YOLOv7-BRA (YOLOv7 with Bi-level Routing Attention ). We identified eight different behavior patterns, including standing, sitting, speaking, listening, walking, raising hands, reading, and writing. We constructed a dataset, which contained 11,248 labels and 4,001 images, with an emphasis on the common behavior of raising hands in a classroom setting (Student Classroom Behavior dataset, SCB-Dataset). To improve detection accuracy, we added the biformer attention module to the YOLOv7 network. Finally, we fused the results from YOLOv7 CrowdHuman, SlowFast, and DeepSort models to obtain student classroom behavior data. We conducted experiments on the SCB-Dataset, and YOLOv7-BRA achieved an mAP@0.5 of 87.1%, resulting in a 2.2% improvement over previous results. Our SCB-dataset can be downloaded from: https://github.com/Whiffe/SCB-datase
Paper Structure (17 sections, 4 equations, 12 figures, 1 table)

This paper contains 17 sections, 4 equations, 12 figures, 1 table.

Table of Contents

  1. Introduction
  2. Our main contributions are as follows
  3. Related word
  4. Student classroom behavior dataset
  5. Students classroom behavior detection
  6. Attention mechanisms
  7. Student Behavior Detection System
  8. SCB-dataset
  9. YOLOv7+BRA
  10. Student Classroom Behavior Detection System
  11. Experiment and Analysis
  12. Experimental Environment and Dataset
  13. Experimental content
  14. Model Training
  15. Evaluation Metrics
  16. ...and 2 more sections

Figures (12)

  • Figure 1: YOLOv7 and YOLOv7-BRA detection results comparison. It is clear that YOLOv7-BRA has better detection performance.
  • Figure 2: Challenges in the student hand-raising behavior dataset include dense environments, similar behaviors, and pixel differences.
  • Figure 3: Challenges in the student hand-raising behavior dataset include varying shooting angles, class differences, and different learning stages.
  • Figure 4: Statistical Analysis of the Number of Hand-Raisings in the SCB-dataset.
  • Figure 5: Bi-level Routing Attention.
  • ...and 7 more figures