Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pose-invariant face recognition via feature-space pose frontalization

Nikolay Stanishev, Yuhang Lu, Touradj Ebrahimi

TL;DR

This work tackles pose-invariant face recognition by performing frontalization in feature space rather than image space, via a novel Feature Space Pose Frontalization Module (FSPFM) that uses full pose embeddings as soft gates. A two-stage training regime—pre-training followed by attention-guided fine-tuning with a dedicated domain-adaptation loss—bridges the gap between frontal and profile representations. Empirical results across five benchmarks show state-of-the-art performance in cross-pose scenarios (notably CPLFW and CFP-FP) and solid gains on standard datasets, validating the effectiveness of feature-space frontalization and the proposed attention mechanism. The approach offers a robust, computationally efficient path for real-world PIFR applications by avoiding extensive image-space synthesis while leveraging pose information for residual learning.

Abstract

Pose-invariant face recognition has become a challenging problem for modern AI-based face recognition systems. It aims at matching a profile face captured in the wild with a frontal face registered in a database. Existing methods perform face frontalization via either generative models or learning a pose robust feature representation. In this paper, a new method is presented to perform face frontalization and recognition within the feature space. First, a novel feature space pose frontalization module (FSPFM) is proposed to transform profile images with arbitrary angles into frontal counterparts. Second, a new training paradigm is proposed to maximize the potential of FSPFM and boost its performance. The latter consists of a pre-training and an attention-guided fine-tuning stage. Moreover, extensive experiments have been conducted on five popular face recognition benchmarks. Results show that not only our method outperforms the state-of-the-art in the pose-invariant face recognition task but also maintains superior performance in other standard scenarios.

Pose-invariant face recognition via feature-space pose frontalization

TL;DR

This work tackles pose-invariant face recognition by performing frontalization in feature space rather than image space, via a novel Feature Space Pose Frontalization Module (FSPFM) that uses full pose embeddings as soft gates. A two-stage training regime—pre-training followed by attention-guided fine-tuning with a dedicated domain-adaptation loss—bridges the gap between frontal and profile representations. Empirical results across five benchmarks show state-of-the-art performance in cross-pose scenarios (notably CPLFW and CFP-FP) and solid gains on standard datasets, validating the effectiveness of feature-space frontalization and the proposed attention mechanism. The approach offers a robust, computationally efficient path for real-world PIFR applications by avoiding extensive image-space synthesis while leveraging pose information for residual learning.

Abstract

Pose-invariant face recognition has become a challenging problem for modern AI-based face recognition systems. It aims at matching a profile face captured in the wild with a frontal face registered in a database. Existing methods perform face frontalization via either generative models or learning a pose robust feature representation. In this paper, a new method is presented to perform face frontalization and recognition within the feature space. First, a novel feature space pose frontalization module (FSPFM) is proposed to transform profile images with arbitrary angles into frontal counterparts. Second, a new training paradigm is proposed to maximize the potential of FSPFM and boost its performance. The latter consists of a pre-training and an attention-guided fine-tuning stage. Moreover, extensive experiments have been conducted on five popular face recognition benchmarks. Results show that not only our method outperforms the state-of-the-art in the pose-invariant face recognition task but also maintains superior performance in other standard scenarios.

Paper Structure

This paper contains 17 sections, 3 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Face frontalization-based method
  4. Pose-invariant feature representation-based method
  5. Proposed Method
  6. Feature space pose frontalization module (FSPFM)
  7. Frontal-profile feature adaptation
  8. Loss functions
  9. Experimental Results
  10. Experimental setups
  11. Datasets
  12. Implementation details
  13. Experimental results
  14. Performance on multiple datasets
  15. Comparison to state-of-the-art PIFR methods
  16. ...and 2 more sections

Figures (1)

  • Figure 1: Pipeline of the proposed PIFR method. It consists of two stages of optimization, namely pre-training and fine-tuning. Consequently, the P-Net is regarded as the target model to perform PIFR.