PETALface: Parameter Efficient Transfer Learning for Low-resolution Face Recognition

TL;DR

This work tackles low-resolution face recognition by addressing catastrophic forgetting and the gallery-probe domain gap through PETALface, a parameter-efficient transfer learning approach. PETALface introduces two twin LoRA blocks that act as proxy encoders for high- and low-resolution data, weighted by per-image quality scores and integrated with a shared final embedding. It achieves strong LR-FR performance with only 0.48% of parameters trainable, while preserving HR and mixed-quality performance, validated on TinyFace, BRIAR, and IJB-S benchmarks. The results demonstrate robust generalization and highlight the importance of image-quality-guided, parameter-efficient adaptation for surveillance-grade face recognition.

Abstract

Pre-training on large-scale datasets and utilizing margin-based loss functions have been highly successful in training models for high-resolution face recognition. However, these models struggle with low-resolution face datasets, in which the faces lack the facial attributes necessary for distinguishing different faces. Full fine-tuning on low-resolution datasets, a naive method for adapting the model, yields inferior performance due to catastrophic forgetting of pre-trained knowledge. Additionally the domain difference between high-resolution (HR) gallery images and low-resolution (LR) probe images in low resolution datasets leads to poor convergence for a single model to adapt to both gallery and probe after fine-tuning. To this end, we propose PETALface, a Parameter-Efficient Transfer Learning approach for low-resolution face recognition. Through PETALface, we attempt to solve both the aforementioned problems. (1) We solve catastrophic forgetting by leveraging the power of parameter efficient fine-tuning(PEFT). (2) We introduce two low-rank adaptation modules to the backbone, with weights adjusted based on the input image quality to account for the difference in quality for the gallery and probe images. To the best of our knowledge, PETALface is the first work leveraging the powers of PEFT for low resolution face recognition. Extensive experiments demonstrate that the proposed method outperforms full fine-tuning on low-resolution datasets while preserving performance on high-resolution and mixed-quality datasets, all while using only 0.48% of the parameters. Code: https://kartik-3004.github.io/PETALface/

Figures (5)

• Figure 1: The proposed PETALface: a parameter efficient transfer learning approach adapts to low-resolution datasets beating the performance of pre-trained models with negligible drop in performance on high-resolution and mixed-quality datasets. PETALface enables development of generalized models achieving competitive performance on high-resolution (LFW, CFP-FP, CPLFW, AgeDB, CALFW, CFP-FF) and mixed-quality datasets (IJB-B, IJB-C), with big enhancements on low-quality and surveillance quality datasets (TinyFace, BRIAR, IJB-S).
• Figure 2: (a) An illustration of the gallery and probe images from low-resolution dataset (BRIAR). Gallery images usually are high quality compared to the probe images. (b) t-SNE plot for the gallery and probe images of the BRIAR dataset. (c) Average CNN-IQA scores of gallery and probe images for 50 identities of the BRIAR dataset.
• Figure 3: Overview of the proposed PETALface. We include an additional trainable module in linear layers present in attention layers and the final feature projection MLP. The trainable module is highlighted on the right. Specifically, we add two LoRA layers, where the weightage $\alpha$ is decided based on the input-image quality, computed using an off-the-shelf image quality assessment network (IQA).
• Figure 4: Comparison of initial gradients when (a) Full fine-tuning a model and using (b) PETALface fine-tuning approach. We can see that PETALface has small initial gradients which results in stable and gradual convergence. NOTE: The scale of the 'Gradient Values' axis for Full fine-tuning and PETALface is different.
• Figure 5: Failure Case Analysis of PETALface on the BRIAR dataset. All the subjects are consented for publication.