CLFace: A Scalable and Resource-Efficient Continual Learning Framework for Lifelong Face Recognition

TL;DR

CLFace is introduced, a continual learning framework designed to preserve and incrementally extend the learned knowledge that outperforms baseline approaches and state-of-the-art methods on unseen identities using both in-domain and out-of-domain datasets.

Abstract

An important aspect of deploying face recognition (FR) algorithms in real-world applications is their ability to learn new face identities from a continuous data stream. However, the online training of existing deep neural network-based FR algorithms, which are pre-trained offline on large-scale stationary datasets, encounter two major challenges: (I) catastrophic forgetting of previously learned identities, and (II) the need to store past data for complete retraining from scratch, leading to significant storage constraints and privacy concerns. In this paper, we introduce CLFace, a continual learning framework designed to preserve and incrementally extend the learned knowledge. CLFace eliminates the classification layer, resulting in a resource-efficient FR model that remains fixed throughout lifelong learning and provides label-free supervision to a student model, making it suitable for open-set face recognition during incremental steps. We introduce an objective function that employs feature-level distillation to reduce drift between feature maps of the student and teacher models across multiple stages. Additionally, it incorporates a geometry-preserving distillation scheme to maintain the orientation of the teacher model's feature embedding. Furthermore, a contrastive knowledge distillation is incorporated to continually enhance the discriminative power of the feature representation by matching similarities between new identities. Experiments on several benchmark FR datasets demonstrate that CLFace outperforms baseline approaches and state-of-the-art methods on unseen identities using both in-domain and out-of-domain datasets.

Figures (3)

• Figure 1: Overview of the training and testing protocol of the proposed CLFace algorithm. Here, we adopt a more challenging setting by testing CLFace on unseen identities from both in-domain and out-of-domain datasets, to evaluate its robustness in real-world biometric applications. This contrasts with typical CL protocols that evaluate models on seen identities within the same dataset.
• Figure 2: Overview of the proposed CLFace framework for scalable, resource-efficient lifelong face recognition. It consists of a teacher model $M_{t-1}$ trained on previously seen identities, and a student model $M_t$ initialized from $M_{t-1}$. When a new face identity ($x_{i}^{t}$) is fed to both models, the MSFD loss penalizes the drift in local features between the student and teacher models, helping to retain learned feature representations. GPKD constrains the student model to maintain a similar orientation to the teacher model's embedding space, while CKD enhances the discriminative power of the feature representation through similarity matching between new identities.
• Figure 3: Comparison of CLFace with other approaches for evaluating in-domain performance in (a) 10-step and (b) 5-step learning scenarios.