Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond the Visible: A Survey on Cross-spectral Face Recognition

David Anghelone, Cunjian Chen, Arun Ross, Antitza Dantcheva

TL;DR

This survey addresses cross-spectral face recognition (CFR), focusing on the challenge of matching faces across visible and infrared modalities. It surveys deep learning approaches that either learn domain-invariant features or synthesize cross-spectral images to enable visible-face recognition, emphasizing NIR-to-VIS performance while detailing SWIR, MWIR, LWIR, and polarimetric variants. The authors catalog datasets, analyze loss functions, and compare reflective versus emissive IR strategies, highlighting how polarization and advanced generative models improve robustness. The work concludes with a forward-looking view on open challenges, the importance of larger, diverse datasets, and the potential shift toward diffusion-based synthesis and rigorous theoretical underpinnings to advance CFR in security- and defense-relevant scenarios.

Abstract

Cross-spectral face recognition (CFR) refers to recognizing individuals using face images stemming from different spectral bands, such as infrared versus visible. While CFR is inherently more challenging than classical face recognition due to significant variation in facial appearance caused by the modality gap, it is useful in many scenarios including night-vision biometrics and detecting presentation attacks. Recent advances in deep neural networks (DNNs) have resulted in significant improvement in the performance of CFR systems. Given these developments, the contributions of this survey are three-fold. First, we provide an overview of CFR, by formalizing the CFR problem and presenting related applications. Secondly, we discuss the appropriate spectral bands for face recognition and discuss recent CFR methods, placing emphasis on deep neural networks. In particular we describe techniques that have been proposed to extract and compare heterogeneous features emerging from different spectral bands. We also discuss the datasets that have been used for evaluating CFR methods. Finally, we discuss the challenges and future lines of research on this topic.

Beyond the Visible: A Survey on Cross-spectral Face Recognition

TL;DR

This survey addresses cross-spectral face recognition (CFR), focusing on the challenge of matching faces across visible and infrared modalities. It surveys deep learning approaches that either learn domain-invariant features or synthesize cross-spectral images to enable visible-face recognition, emphasizing NIR-to-VIS performance while detailing SWIR, MWIR, LWIR, and polarimetric variants. The authors catalog datasets, analyze loss functions, and compare reflective versus emissive IR strategies, highlighting how polarization and advanced generative models improve robustness. The work concludes with a forward-looking view on open challenges, the importance of larger, diverse datasets, and the potential shift toward diffusion-based synthesis and rigorous theoretical underpinnings to advance CFR in security- and defense-relevant scenarios.

Abstract

Cross-spectral face recognition (CFR) refers to recognizing individuals using face images stemming from different spectral bands, such as infrared versus visible. While CFR is inherently more challenging than classical face recognition due to significant variation in facial appearance caused by the modality gap, it is useful in many scenarios including night-vision biometrics and detecting presentation attacks. Recent advances in deep neural networks (DNNs) have resulted in significant improvement in the performance of CFR systems. Given these developments, the contributions of this survey are three-fold. First, we provide an overview of CFR, by formalizing the CFR problem and presenting related applications. Secondly, we discuss the appropriate spectral bands for face recognition and discuss recent CFR methods, placing emphasis on deep neural networks. In particular we describe techniques that have been proposed to extract and compare heterogeneous features emerging from different spectral bands. We also discuss the datasets that have been used for evaluating CFR methods. Finally, we discuss the challenges and future lines of research on this topic.
Paper Structure (30 sections, 27 equations, 22 figures, 10 tables)

This paper contains 30 sections, 27 equations, 22 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Formalization of CFR
  3. Applications of CFR
  4. Law Enforcement and Homeland Security
  5. Military Applications
  6. Commercial Applications
  7. Presentation Attack Detection
  8. Sketch-Visual Identification
  9. Different Spectra
  10. Active IR
  11. Passive IR
  12. Polarimetric Thermal Imagery
  13. Cross-spectral Face Recognition
  14. Face Preprocessing
  15. Deep Feature Extraction
  16. ...and 15 more sections

Figures (22)

  • Figure 1: Examples of heterogeneous face images of the same subject captured in different modalities, i.e., domains. Images are from the MCXFace dataset George_IEEETIFS_2022.
  • Figure 2: An overview of cross-spectral face recognition
  • Figure 3: A monitoring system equipped with a thermal sensor applies face and facial landmark detection to a video sequence captured in the wild at night, successfully detecting the person despite challenging conditions, paving the way for subsequent identification steps. Images are from Anghelone et al.anghelone2022tfld.
  • Figure 4: Electromagnetic spectrum. Modalities and their associated wavelengths, highlighting the visible and infrared radiations.
  • Figure 5: Face beyond the visible. A comparison of a face in visible (VIS) and infrared bands, viz., NIR, SWIR, MWIR and LWIR. We note the different physiological properties in both Active IR (NIR, SWIR) and Passive IR (MWIR, LWIR) bands. Figure credit: Hu al.hu2017heterogeneous
  • ...and 17 more figures