ViTNT-FIQA: Training-Free Face Image Quality Assessment with Vision Transformers

TL;DR

This work introduces ViTNT-FIQA, a training-free method for face image quality assessment that leverages the stability of patch embeddings across intermediate ViT blocks. By computing cross-block distances between L2-normalized patch embeddings and mapping them to per-patch scores, then aggregating with either uniform or attention-weighted schemes, the approach yields a robust image-level quality score from a single forward pass. Extensive ablations show that early transformer blocks carry strong quality signals, and attention-weighted aggregation enhances performance; the method generalizes across different ViT models and FR tasks, achieving competitive results against state-of-the-art FIQA while avoiding training, backpropagation, or architectural changes. Practically, ViTNT-FIQA offers a fast, model-agnostic quality assessment tool that can be deployed with pre-trained ViT-based face recognition systems, improving reliability in verification pipelines without additional training overhead.

Abstract

Face Image Quality Assessment (FIQA) is essential for reliable face recognition systems. Current approaches primarily exploit only final-layer representations, while training-free methods require multiple forward passes or backpropagation. We propose ViTNT-FIQA, a training-free approach that measures the stability of patch embedding evolution across intermediate Vision Transformer (ViT) blocks. We demonstrate that high-quality face images exhibit stable feature refinement trajectories across blocks, while degraded images show erratic transformations. Our method computes Euclidean distances between L2-normalized patch embeddings from consecutive transformer blocks and aggregates them into image-level quality scores. We empirically validate this correlation on a quality-labeled synthetic dataset with controlled degradation levels. Unlike existing training-free approaches, ViTNT-FIQA requires only a single forward pass without backpropagation or architectural modifications. Through extensive evaluation on eight benchmarks (LFW, AgeDB-30, CFP-FP, CALFW, Adience, CPLFW, XQLFW, IJB-C), we show that ViTNT-FIQA achieves competitive performance with state-of-the-art methods while maintaining computational efficiency and immediate applicability to any pre-trained ViT-based face recognition model.

Figures (10)

• Figure 1: Boxplots of mean L2 distances between corresponding patch embeddings from consecutive ViT-B blocks computed for 11 quality groups, each having 0.5M images, from 5.5M images of SynFIQA mrfiqa. Each box summarizes the distribution of average patch-embedding distances across images in a quality group, lower distances empirically correspond to higher ground-truth quality for most block transitions. The inset (Block 11 $\leftrightarrow$ 12) shows the quality gradient (low $\rightarrow$ high) and illustrates how the distances across groups provide a measure of quality discriminability, i.e. the higher the quality, the lower the distance.
• Figure 2: Overview of our ViT-based quality assessment method ViTNT-FIQA. (1) The face image is patchified and embedded. (2) Intermediate patch representations are extracted from selected transformer blocks. (3) L2-normalized embeddings are compared across consecutive blocks to measure patch-level feature distances. (4) Distances are mapped to quality scores per patch level, which are aggregated, uniformly or using attention weights, to produce the final image-level quality estimate.
• Figure 3: Error-versus-Discard Characteristic (EDC) curves for FNMR@FMR=$1e-3$ of our proposed method in comparison to SOTA. Results shown on eight benchmark datasets: LFW LFWTech, AgeDB-30 agedb, CFP-FP cfp-fp, CALFW CALFW, Adience Adience, CPLFW CPLFWTech, XQLFW XQLFW, and IJB-C ijbc, using ArcFace deng2019arcface, ElasticFace elasticface, MagFace MagFace, and CurricularFace curricularFace FR models. Our method ViTNT-FIQA is marked with the red line.
• Figure 4: Boxplots of mean L2 distances between corresponding patch embeddings from consecutive ViT-S blocks computed for 11 quality groups, each having 0.5M images, from 5.5M images of SynFIQA mrfiqa. Each box summarizes the distribution of average patch-embedding distances across images in a quality group, lower distances empirically correspond to higher ground-truth quality for most block transitions, i.e. the higher the quality, the lower the distance.
• Figure 5: Comprehensive ablation analysis via Error-versus-Discard Characteristic (EDC) curves at FMR=$1e-2$. Each column represents one of five ablation studies: Dataset (generalization across WebFace4M, WebFace12M, CLIP, FRoundation), Architecture (ViT-S vs ViT-B depth comparison), Block Depth (computational trade-offs from 4 to 24 blocks), Attention (last-block vs all-blocks aggregation at varying depths), and Block Windows (consecutive 6-block segments from early to late network stages). Each row shows results on a different benchmark dataset (AgeDB-30, CALFW, CFP-FP, CPLFW, LFW, XQLFW). The Dataset study confirms cross-model generalization with FR-trained models (WebFace4M, WebFace12M) outperforming foundation models (CLIP, FRoundation). The Architecture study reveals minimal performance gap between ViT-S and ViT-B, validating depth-independence. The Block Depth study demonstrates that 12-20 blocks provide optimal efficiency-performance balance, with diminishing returns beyond 16 blocks. The Attention study shows consistent improvements from attention-weighting, particularly at 12-20 block depths. The Block Windows study reveals that early transformer blocks (0-5) capture the strongest quality signals. All curves use ArcFace for cross-model evaluation. Across all studies, FNMR decreases steadily as low-quality samples are discarded, validating ViTNT-FIQA's effectiveness in identifying quality-degraded images. The consistent color coding highlights method performance: WebFace4M-based configurations (blue) serve as the primary baseline across multiple studies.