Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors

TL;DR

Vec2Face presents a scalable method to synthesize large-scale, identity-aware face datasets by mapping random high-dimensional vectors to face images. It combines a feature masked autoencoder with an image decoder and a GAN discriminator to preserve identity while enabling controlled inter-class separation and intra-class variation, and introduces AttrOP to steer attributes during generation. The approach achieves state-of-the-art FR performance on multiple real benchmarks with synthetic data, and scaling to 15 million images yields further improvements, even surpassing real data at the same scale on several tests. This technique offers a practical, privacy-friendly solution for building large FR datasets and can be extended to other object domains.

Abstract

This paper studies how to synthesize face images of non-existent persons, to create a dataset that allows effective training of face recognition (FR) models. Besides generating realistic face images, two other important goals are: 1) the ability to generate a large number of distinct identities (inter-class separation), and 2) a proper variation in appearance of the images for each identity (intra-class variation). However, existing works 1) are typically limited in how many well-separated identities can be generated and 2) either neglect or use an external model for attribute augmentation. We propose Vec2Face, a holistic model that uses only a sampled vector as input and can flexibly generate and control the identity of face images and their attributes. Composed of a feature masked autoencoder and an image decoder, Vec2Face is supervised by face image reconstruction and can be conveniently used in inference. Using vectors with low similarity among themselves as inputs, Vec2Face generates well-separated identities. Randomly perturbing an input identity vector within a small range allows Vec2Face to generate faces of the same identity with proper variation in face attributes. It is also possible to generate images with designated attributes by adjusting vector values with a gradient descent method. Vec2Face has efficiently synthesized as many as 300K identities, whereas 60K is the largest number of identities created in the previous works. As for performance, FR models trained with the generated HSFace datasets, from 10k to 300k identities, achieve state-of-the-art accuracy, from 92% to 93.52%, on five real-world test sets (\emph{i.e.}, LFW, CFP-FP, AgeDB-30, CALFW, and CPLFW). For the first time, the FR model trained using our synthetic training set achieves higher accuracy than that trained using a same-scale training set of real face images on the CALFW, IJBB, and IJBC test sets.

Figures (20)

• Figure 1: Example images generated by Vec2Face. From a random vector, we generate a face (ID image). We then perturb this vector with random values to generate diverse face images. Larger perturbation added to this vector results in larger dissimilarity to the ID images. The images in the frame are likely of the same people as the ID image.
• Figure 2: Architecture of Vec2Face. Given a real face image, we compute its feature "IM feature" using a face recognition model. This feature is projected and expanded into a feature map, and the latter is processed by a feature masked autoencoder (fMAE). Inside the fMAE, the rows in the feature map are randomly masked out before being processed by the encoder. The projected image feature is then used to form the full-size feature map before being processed by the decoder. Finally, the decoder outputs a feature map. Outside the fMAE, a small image decoder reconstructs the pixels in the original image based on the output of fMAE. During inference, Vec2Face accepts a randomized vector and generates a face image. This process has properties demonstrated in Fig. \ref{['fig:teaser']}.
• Figure 3: Reconstruction results for training (left) and unseen (right) identities. Specifically, we extract features of original images by using a FR model and feed them to Vec2Face for reconstruction. We observe that the reconstructed images still maintain the same identity while removing some image borders and backgrounds and transferring sketches into photo-realistic images.
• Figure 4: AttrOP
• Figure 5: Comparing existing synthetic FR dataset generation methods on inter-class separability.