Quality-Aware Prototype Memory for Face Representation Learning

TL;DR

Quality-Aware Prototype Memory addresses the sensitivity of Prototype Memory to low-quality face images by introducing quality-weighted prototype generation. The method computes new prototypes as $p_{new} = F_{norm}\left(\frac{\sum_{j=1}^{k} q_j \mathbf{x}_j}{\sum_{j=1}^{k} q_j}\right)$, where $q_j$ encodes face quality, thereby reducing the influence of poor-quality embeddings on the prototype placement. The authors explore several quality-estimation strategies, including feature-norm proxies $q_j = \frac{\|\mathbf{x}_j\|}{\|\mathbf{x}\|_{max}}$ and recognizability-based scores derived from a dedicated unrecognizable identity prototype, and demonstrate improvements on multiple face-recognition benchmarks with both medium and large backbone networks. Across datasets such as CFP-FP, AgeDB-30, CALFW, CPLFW, XQLFW, and RFW, QA-PM consistently outperforms the original Prototype Memory, with soft recognizability scores often delivering the best results. The work offers a practical, low-overhead enhancement for large-scale face representation learning by integrating quality estimation into prototype-based training signals.

Abstract

Prototype Memory is a powerful model for face representation learning. It enables training face recognition models on datasets of any size by generating prototypes (classifier weights) on the fly and efficiently utilizing them. Prototype Memory demonstrated strong results in many face recognition benchmarks. However, the algorithm of prototype generation, used in it, is prone to the problems of imperfectly calculated prototypes in case of low-quality or poorly recognizable faces in the images, selected for the prototype creation. All images of the same person presented in the mini-batch are used with equal weights, and the resulting averaged prototype can be contaminated by imperfect embeddings of low-quality face images. This may lead to misleading training signals and degrade the performance of the trained models. In this paper, we propose a simple and effective way to improve Prototype Memory with quality-aware prototype generation. Quality-Aware Prototype Memory uses different weights for images of different quality in the process of prototype generation. With this improvement, prototypes receive more informative signals from high-quality images and are less affected by low-quality ones. We propose and compare several methods of quality estimation and usage, perform extensive experiments on the different face recognition benchmarks and demonstrate the advantages of the proposed model compared to the basic version of Prototype Memory.

Figures (2)

• Figure 1: Prototype generation with (a) Prototype Memory, (b) Quality-Aware Prototype Memory
• Figure 2: Example prototypes, generated with (a) Prototype Memory, (b) Quality-Aware Prototype Memory.