A Test Suite for Efficient Robustness Evaluation of Face Recognition Systems

TL;DR

The work tackles robustness evaluation for face recognition by addressing the inefficiency of pursuing extensive adversarial attacks and the impracticality of formal Lipschitz guarantees. It introduces RobFace, a search-free, system-agnostic framework that uses a pre-optimised test suite of transferable adversarial samples to rapidly estimate robustness across multiple perturbation spaces $p$-norms and natural transformations. The key contributions include constructing RobFace-01, demonstrating strong Pearson correlations ($r \approx 0.90$–$0.99$) with reference robustness across unseen systems, and achieving up to ~200x speedups while maintaining broad perturbation coverage and defense against adaptive attacks via seed randomisation. This approach enables scalable, agile robustness assessment in deployment pipelines and can be extended to other recognition tasks beyond face recognition.

Abstract

Face recognition is a widely used authentication technology in practice, where robustness is required. It is thus essential to have an efficient and easy-to-use method for evaluating the robustness of (possibly third-party) trained face recognition systems. Existing approaches to evaluating the robustness of face recognition systems are either based on empirical evaluation (e.g., measuring attacking success rate using state-of-the-art attacking methods) or formal analysis (e.g., measuring the Lipschitz constant). While the former demands significant user efforts and expertise, the latter is extremely time-consuming. In pursuit of a comprehensive, efficient, easy-to-use and scalable estimation of the robustness of face recognition systems, we take an old-school alternative approach and introduce RobFace, i.e., evaluation using an optimised test suite. It contains transferable adversarial face images that are designed to comprehensively evaluate a face recognition system's robustness along a variety of dimensions. RobFace is system-agnostic and still consistent with system-specific empirical evaluation or formal analysis. We support this claim through extensive experimental results with various perturbations on multiple face recognition systems. To our knowledge, RobFace is the first system-agnostic robustness estimation test suite.

Figures (7)

• Figure 1: Scatter plot of original input $x$ vs perturbed examples $x'$ with colour coding reflecting the effectiveness of the perturbation. Full red dots indicate perturbations that alter predictions for all systems, mixed red and black dots indicate those that alter predictions of only a limited number of systems, and full black dots indicate no effect on altering predictions.
• Figure 2: Displayed above are three face images, each with a resolution of 250$\times$250 pixels. In face recognition, an input could look like $\bm{x}_1 = (\textnormal{im}_\textnormal{a}, \textnormal{im}_\textnormal{b})$ or $\bm{x}_2 = (\textnormal{im}_\textnormal{b}, \textnormal{im}_\textnormal{c})$. Here, $n=6.25\time 10^4$. Correct predictions of a classifier $h$ should be $h(\bm{x}_1) = 0$ (different persons) and $h(\bm{x}_2) = 1$ (same person).
• Figure 3: Differences in robustness estimating space. The left shows the process of evaluating robustness using a test suite (i.e., what we propose in this work). The right shows the process of empirical evaluation using adversarial attacks, which require us to iteratively approximate the maximum change in the output through optimisation (shown as a loop).
• Figure 4: Illustration of various face image perturbations, including perturbation within certain $p-$norm bounds and other perturbations such as wearing unique accessories, altering ambient light or background, and adjusting camera angle. Some transformations have additional restrictions such as illumination intensity and age range.
• Figure 5: We show that the proposed approach, RobFace, correlates well with the reference robust accuracies, with Pearson correlation ranging from 0.9 to 0.99. A random group of face systems is used for tuning, and the rest are for testing. Indices of the testing systems are underlined. Reference-1 is the robust accuracy by searching adversarial examples via PGD madry2017towards. Reference-2 is CLEVER weng2018evaluating, a theoretical robustness evaluation approach.

Theorems & Definitions (4)

• Example 1
• Example 2
• Example 3
• Example 4