Impact of Face Alignment on Face Image Quality

TL;DR

This study investigates how face alignment influences face image quality assessment (FIQA) by comparing Cropped versus Aligned face images across LFW, IJB-B, and SCFace using two detectors (MTCNN and RetinaFace) and four FIQA methods (SER-FIQ, FaceQAN, DifFIQA, SDD-FIQA). Alignment is formalized as a similarity transform $T$ learned from landmarks, with $T$ minimizing $\arg\min_T \sum_i \|T(p_i) - q_i\|^2$ and parameters $α$, $θ$, $t_x$, $t_y$, producing results that highlight how misalignment reduces FIQA scores, especially under surveillance-like conditions where image quality is low. The findings show consistent quality-score degradation for Cropped images and improved scores for Aligned images, with the largest effects in SCFace and with methods most sensitive to alignment (e.g., SER-FIQ, FaceQAN). The work underscores the need to consider alignment robustness in FIQA development and evaluation, as FIQA models are often trained on aligned data and may not generalize well to misaligned inputs. Formally incorporating alignment into FIQA design could improve reliability in real-world biometric systems, particularly under challenging conditions.

Abstract

Face alignment is a crucial step in preparing face images for feature extraction in facial analysis tasks. For applications such as face recognition, facial expression recognition, and facial attribute classification, alignment is widely utilized during both training and inference to standardize the positions of key landmarks in the face. It is well known that the application and method of face alignment significantly affect the performance of facial analysis models. However, the impact of alignment on face image quality has not been thoroughly investigated. Current FIQA studies often assume alignment as a prerequisite but do not explicitly evaluate how alignment affects quality metrics, especially with the advent of modern deep learning-based detectors that integrate detection and landmark localization. To address this need, our study examines the impact of face alignment on face image quality scores. We conducted experiments on the LFW, IJB-B, and SCFace datasets, employing MTCNN and RetinaFace models for face detection and alignment. To evaluate face image quality, we utilized several assessment methods, including SER-FIQ, FaceQAN, DifFIQA, and SDD-FIQA. Our analysis included examining quality score distributions for the LFW and IJB-B datasets and analyzing average quality scores at varying distances in the SCFace dataset. Our findings reveal that face image quality assessment methods are sensitive to alignment. Moreover, this sensitivity increases under challenging real-life conditions, highlighting the importance of evaluating alignment's role in quality assessment.

Figures (5)

• Figure 1: The general overview of our pipeline is as follows. A face detector first processes an original image containing a face. The detector outputs a bounding box (red dotted frame) and facial landmarks (blue dots). The image is then cropped to extract the face region, resulting in what we refer to as the "Cropped Image." In parallel, the input image is aligned based on the detected landmarks, producing the "Aligned Image." This alignment standardizes the face image, such as ensuring the line connecting the eyes (green dotted line) is parallel to the horizontal axis. Finally, both the cropped and aligned images are evaluated using a FIQA method to estimate their FIQA scores.
• Figure 2: Sample original images from LFW LFW, IJB-B IJB-B, and SCFace SCFace.
• Figure 3: Sample original images from LFW LFW, IJB-B IJB-B, and SCFace SCFace, their cropped and aligned versions. The results of each FIQA method are written below in each cropped and aligned image.
• Figure 4: Quality score distributions for LFW LFW dataset.
• Figure 5: Quality score distributions for IJB-B IJB-B dataset.