Use of triplet loss for facial restoration in low-resolution images

TL;DR

A novel SR model FTLGAN is proposed, which focuses on generating high-resolution images that preserve individual identities rather than merely improving image quality, thereby maximizing the performance of FR models and addressing the challenge of improving classification performance in facial recognition systems.

Abstract

In recent years, facial recognition (FR) models have become the most widely used biometric tool, achieving impressive results on numerous datasets. However, inherent hardware challenges or shooting distances often result in low-resolution images, which significantly impact the performance of FR models. To address this issue, several solutions have been proposed, including super-resolution (SR) models that generate highly realistic faces. Despite these efforts, significant improvements in FR algorithms have not been achieved. We propose a novel SR model FTLGAN, which focuses on generating high-resolution images that preserve individual identities rather than merely improving image quality, thereby maximizing the performance of FR models. The results are compelling, demonstrating a mean value of d' 21% above the best current state-of-the-art models, specifically having a value of d' = 1.099 and AUC = 0.78 for 14x14 pixels, d' = 2.112 and AUC = 0.92 for 28x28 pixels, and d' = 3.049 and AUC = 0.98 for 56x56 pixels. The contributions of this study are significant in several key areas. Firstly, a notable improvement in facial recognition performance has been achieved in low-resolution images, specifically at resolutions of 14x14, 28x28, and 56x56 pixels. Secondly, the enhancements demonstrated by FTLGAN show a consistent response across all resolutions, delivering outstanding performance uniformly, unlike other comparative models. Thirdly, an innovative approach has been implemented using triplet loss logic, enabling the training of the super-resolution model solely with real images, contrasting with current models, and expanding potential real-world applications. Lastly, this study introduces a novel model that specifically addresses the challenge of improving classification performance in facial recognition systems by integrating facial recognition quality as a loss during model training.

Figures (8)

• Figure 1: Visual comparison of three models in the process of super-resolution between 3 models, taking an image from $14\times14$ to $112\times112$ pixels. In a) results of a bicubic interpolation, in b) the results of the Real-SRGAN model wang2021realesrgantrainingrealworldblind, and in c) the GFPGAN model.
• Figure 2: Explanatory diagram of the face re-identification task: an example of two faces that do not belong to the same identity is presented. In this case $d < \theta$.
• Figure 3: Example of a curve of genuine and impostors, using Euclidean distance between pairs, where the two curves should be as far apart as possible to improve task performance.
• Figure 4: Example of a roc where the FMR can be seen on the x-axis and 1-FNMR on the y-axis. The blue line corresponds to the ROC curve while the red curve corresponds to an ROC with AUC 0.5 showing randomness or total confusion between classes.
• Figure 5: Triplet loss training architecture applied to the superresolution process, where T(A,N,P) corresponds to the triplet loss function applied on the vector of characteristics generated for the anchor image, negative and positive correspondingly.