Few-Shot Continual Learning for Activity Recognition in Classroom Surveillance Images

TL;DR

A few-shot continual learning method that combines supervised contrastive learning (SCL) and an adaptive covariance classifier (ACC) that improves the generalization ability of the model, while the ACC module provides a more accurate description of the distribution of new classes.

Abstract

The application of activity recognition in the "AI + Education" field is gaining increasing attention. However, current work mainly focuses on the recognition of activities in manually captured videos and a limited number of activity types, with little attention given to recognizing activities in surveillance images from real classrooms. In real classroom settings, normal teaching activities such as reading, account for a large proportion of samples, while rare non-teaching activities such as eating, continue to appear. This requires a model that can learn non-teaching activities from few samples without forgetting the normal teaching activities, which necessitates fewshot continual learning (FSCL) capability. To address this gap, we constructed a continual learning dataset focused on classroom surveillance image activity recognition called ARIC (Activity Recognition in Classroom). The dataset has advantages such as multiple perspectives, a wide variety of activities, and real-world scenarios, but it also presents challenges like similar activities and imbalanced sample distribution. To overcome these challenges, we designed a few-shot continual learning method that combines supervised contrastive learning (SCL) and an adaptive covariance classifier (ACC). During the base phase, we proposed a SCL approach based on feature augmentation to enhance the model's generalization ability. In the incremental phase, we employed an ACC to more accurately describe the distribution of new classes. Experimental results demonstrate that our method outperforms other existing methods on the ARIC dataset.

Figures (5)

• Figure 1: Monitoring samples from different perspectives.
• Figure 2: Sample distribution of the 32 activity categories.
• Figure 3: Pipline of feature-augmented supervised contrastive learning.
• Figure 4: Qualitative illustration of the adaptive mechanism. Circles represent samples from old classes, and triangles represent samples from new classes. Red circles and triangles indicate misclassified samples, while yellow triangles represent corrected new class samples.
• Figure 5: The t-SNE plots of base class feature distributions after base stage training on the ARIC dataset for different methods: (a) Finetune, (b) FACT, (c) Our.