Federated Learning for Face Recognition via Intra-subject Self-supervised Learning

TL;DR

The paper tackles privacy-preserving, personalized face recognition in a federated setting by eliminating the need to share raw data. It introduces FedFS, which combines intra-subject self-supervised learning with adaptive soft-labels to learn individualized representations without exposing private data, integrated via a FedAvg-style coordination among a public pre-trained model, a global model, and a client-specific personalized model. The learning objective blends intra-subject loss and a regularization term as $F_{total}= \lambda F_{insub}+(1-\lambda)F_{reg}$ with $\lambda=0.7$, and demonstrates superior performance on DigiFace-1M and VGGFace across multiple backbones and participation scenarios. This work advances practical, privacy-conscious face recognition by reducing intra-class variation and enhancing personalization in large-scale federated settings.

Abstract

Federated Learning (FL) for face recognition aggregates locally optimized models from individual clients to construct a generalized face recognition model. However, previous studies present two major challenges: insufficient incorporation of self-supervised learning and the necessity for clients to accommodate multiple subjects. To tackle these limitations, we propose FedFS (Federated Learning for personalized Face recognition via intra-subject Self-supervised learning framework), a novel federated learning architecture tailored to train personalized face recognition models without imposing subjects. Our proposed FedFS comprises two crucial components that leverage aggregated features of the local and global models to cooperate with representations of an off-the-shelf model. These components are (1) adaptive soft label construction, utilizing dot product operations to reformat labels within intra-instances, and (2) intra-subject self-supervised learning, employing cosine similarity operations to strengthen robust intra-subject representations. Additionally, we introduce a regularization loss to prevent overfitting and ensure the stability of the optimized model. To assess the effectiveness of FedFS, we conduct comprehensive experiments on the DigiFace-1M and VGGFace datasets, demonstrating superior performance compared to previous methods.

Figures (4)

• Figure 1: Pipelines of federated learning-based face recognition methods including our proposed method. (a) The server collects class embedding of client c (e.g. FedFace). (b) Client c continuously downloads public data from the server (e.g. FedFR). (c) Our proposed method(FedFS), client c performs intra-subject self-supervised learning without any additional work.
• Figure 2: (a) is an overview of our proposed training process and (b) is the detailed process of intra-subject loss. The global model outputs two vectors and the personalized model also outputs two vectors. Using each output, we calculate regularization loss and create a $z_c$ vector. Intra-subject loss is measured using the $z_c$ vector and the output vector of the pre-trained model.
• Figure 3: Each graph represents the ROC curve against the pre-trained model @ benchmark and the ROC curve against federated learning methods using the pre-trained model.
• Figure B.1: Average similarity distribution of clients participated in federated learning