Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explainable Face Verification via Feature-Guided Gradient Backpropagation

Yuhang Lu, Zewei Xu, Touradj Ebrahimi

TL;DR

The paper addresses the explainability gap in face verification by introducing Feature-Guided Gradient Backpropagation (FGGB), a gradient-based method that propagates at the feature level to produce both similarity and dissimilarity saliency maps for Accept/Reject decisions. FGGB computes channel-wise gradients $G_A^k = \partial F_A^k / \partial I_A$ with normalization, then weights these maps by the channel-wise cosine similarity $w = (F_A \odot F_B)/(\|F_A\|\|F_B\|)$ to form a robust saliency score $S_A = \sum_{k=1}^N \tilde{G}_A^k \cdot (w_k - \frac{threshold}{N})$, yielding $S_A^+$ (similarity) and $S_A^-$ (dissimilarity). The method is parameter-free, model-agnostic, and demonstrated on ArcFace, AdaFace, and MobileFaceNet, outperforming state-of-the-art XFV approaches in dissimilarity maps while maintaining strong similarity explanations, using Deletion and Insertion metrics on LFW, CPLFW, and CALFW. FGGB’s efficient, gradient-based approach addresses noisy-gradient issues and offers practical, interpretable insights for real-world FR systems. This work lays groundwork for more robust gradient-propagation explanations in vision tasks beyond face verification.

Abstract

Recent years have witnessed significant advancement in face recognition (FR) techniques, with their applications widely spread in people's lives and security-sensitive areas. There is a growing need for reliable interpretations of decisions of such systems. Existing studies relying on various mechanisms have investigated the usage of saliency maps as an explanation approach, but suffer from different limitations. This paper first explores the spatial relationship between face image and its deep representation via gradient backpropagation. Then a new explanation approach FGGB has been conceived, which provides precise and insightful similarity and dissimilarity saliency maps to explain the "Accept" and "Reject" decision of an FR system. Extensive visual presentation and quantitative measurement have shown that FGGB achieves superior performance in both similarity and dissimilarity maps when compared to current state-of-the-art explainable face verification approaches.

Explainable Face Verification via Feature-Guided Gradient Backpropagation

TL;DR

The paper addresses the explainability gap in face verification by introducing Feature-Guided Gradient Backpropagation (FGGB), a gradient-based method that propagates at the feature level to produce both similarity and dissimilarity saliency maps for Accept/Reject decisions. FGGB computes channel-wise gradients with normalization, then weights these maps by the channel-wise cosine similarity to form a robust saliency score , yielding (similarity) and (dissimilarity). The method is parameter-free, model-agnostic, and demonstrated on ArcFace, AdaFace, and MobileFaceNet, outperforming state-of-the-art XFV approaches in dissimilarity maps while maintaining strong similarity explanations, using Deletion and Insertion metrics on LFW, CPLFW, and CALFW. FGGB’s efficient, gradient-based approach addresses noisy-gradient issues and offers practical, interpretable insights for real-world FR systems. This work lays groundwork for more robust gradient-propagation explanations in vision tasks beyond face verification.

Abstract

Recent years have witnessed significant advancement in face recognition (FR) techniques, with their applications widely spread in people's lives and security-sensitive areas. There is a growing need for reliable interpretations of decisions of such systems. Existing studies relying on various mechanisms have investigated the usage of saliency maps as an explanation approach, but suffer from different limitations. This paper first explores the spatial relationship between face image and its deep representation via gradient backpropagation. Then a new explanation approach FGGB has been conceived, which provides precise and insightful similarity and dissimilarity saliency maps to explain the "Accept" and "Reject" decision of an FR system. Extensive visual presentation and quantitative measurement have shown that FGGB achieves superior performance in both similarity and dissimilarity maps when compared to current state-of-the-art explainable face verification approaches.
Paper Structure (11 sections, 5 equations, 3 figures, 3 tables)

This paper contains 11 sections, 5 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Problem Statement
  5. Feature-Guided Gradient Backpropagation (FGGB)
  6. Experimental Results
  7. Implementation Details
  8. Evaluation Methodology
  9. Visual Demonstration
  10. Quantitative Results
  11. Conclusion

Figures (3)

  • Figure 1: Workflow of the proposed Feature-Guided Gradient Backpropagation method. The similarity and dissimilarity maps are calculated respectively given an arbitrary input face pair.
  • Figure 2: Visual comparison of similarity maps generated by FGGB and five other XFV methods. Every two columns represent a pair of genuine faces. The saliency value increases from blue to red color.
  • Figure 3: Visual comparison of dissimilarity maps generated by FGGB and other XFV methods. Every two columns represent a pair of imposter faces. The saliency value increases from blue to red color.