Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comprehensive Survey of Masked Faces: Recognition, Detection, and Unmasking

Mohamed Mahmoud, Mahmoud SalahEldin Kasem, Hyun-Soo Kang

TL;DR

This survey addresses masked-face recognition, detection, and unmasking in the COVID-19 era, detailing challenges such as data scarcity, bias, and occlusion. It synthesizes three DL-centric task families—MFR, FMR, and FU—across real and synthetic datasets, evaluation metrics, and a spectrum of architectures (CNNs, multi-stage detectors, SSD/YOLO variants) and generative restoration methods. Key contributions include a consolidated taxonomy of methods, a catalog of datasets with usage contexts, and a critical discussion of evaluation protocols, biases, and practical deployment considerations. The work highlights the need for richer data, robust synthetic generation, and integrated pipelines to advance reliable masked-face systems in security, healthcare, and daily-life applications.

Abstract

Masked face recognition (MFR) has emerged as a critical domain in biometric identification, especially by the global COVID-19 pandemic, which introduced widespread face masks. This survey paper presents a comprehensive analysis of the challenges and advancements in recognising and detecting individuals with masked faces, which has seen innovative shifts due to the necessity of adapting to new societal norms. Advanced through deep learning techniques, MFR, along with Face Mask Recognition (FMR) and Face Unmasking (FU), represent significant areas of focus. These methods address unique challenges posed by obscured facial features, from fully to partially covered faces. Our comprehensive review delves into the various deep learning-based methodologies developed for MFR, FMR, and FU, highlighting their distinctive challenges and the solutions proposed to overcome them. Additionally, we explore benchmark datasets and evaluation metrics specifically tailored for assessing performance in MFR research. The survey also discusses the substantial obstacles still facing researchers in this field and proposes future directions for the ongoing development of more robust and effective masked face recognition systems. This paper serves as an invaluable resource for researchers and practitioners, offering insights into the evolving landscape of face recognition technologies in the face of global health crises and beyond.

A Comprehensive Survey of Masked Faces: Recognition, Detection, and Unmasking

TL;DR

This survey addresses masked-face recognition, detection, and unmasking in the COVID-19 era, detailing challenges such as data scarcity, bias, and occlusion. It synthesizes three DL-centric task families—MFR, FMR, and FU—across real and synthetic datasets, evaluation metrics, and a spectrum of architectures (CNNs, multi-stage detectors, SSD/YOLO variants) and generative restoration methods. Key contributions include a consolidated taxonomy of methods, a catalog of datasets with usage contexts, and a critical discussion of evaluation protocols, biases, and practical deployment considerations. The work highlights the need for richer data, robust synthetic generation, and integrated pipelines to advance reliable masked-face systems in security, healthcare, and daily-life applications.

Abstract

Masked face recognition (MFR) has emerged as a critical domain in biometric identification, especially by the global COVID-19 pandemic, which introduced widespread face masks. This survey paper presents a comprehensive analysis of the challenges and advancements in recognising and detecting individuals with masked faces, which has seen innovative shifts due to the necessity of adapting to new societal norms. Advanced through deep learning techniques, MFR, along with Face Mask Recognition (FMR) and Face Unmasking (FU), represent significant areas of focus. These methods address unique challenges posed by obscured facial features, from fully to partially covered faces. Our comprehensive review delves into the various deep learning-based methodologies developed for MFR, FMR, and FU, highlighting their distinctive challenges and the solutions proposed to overcome them. Additionally, we explore benchmark datasets and evaluation metrics specifically tailored for assessing performance in MFR research. The survey also discusses the substantial obstacles still facing researchers in this field and proposes future directions for the ongoing development of more robust and effective masked face recognition systems. This paper serves as an invaluable resource for researchers and practitioners, offering insights into the evolving landscape of face recognition technologies in the face of global health crises and beyond.
Paper Structure (21 sections, 8 figures, 8 tables)

This paper contains 21 sections, 8 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Challenges in MFR
  3. Applications of MFR
  4. Related surveys
  5. Masked Face Datasets
  6. Real Mask Datasets
  7. Synthetic Mask Datasets
  8. Evaluation Metrics
  9. Masked Faces Methods
  10. Face Mask Recognition Approaches
  11. Convolutional Neural Network
  12. Multi-stage detection
  13. Single Shot Detector
  14. Transfer Learning
  15. YOLO (You Only Look Once)
  16. ...and 6 more sections

Figures (8)

  • Figure 1: Illustration showcasing the tasks of Masked Face Recognition (MFR), Face Mask Recognition (FMR), and Face Unmasking (FU) with varied outputs for the same input.
  • Figure 2: illustrates the evolving landscape of MFR and FMD studies from 2019 to 2024. The data was sourced from Scopus using keywords 'Masked face recognition' for MFR and 'Face mask detection', 'Face masks', and 'Mask detection' for FMD.
  • Figure 3: Samples of masked & unmasked faces from the Real-Mask Masked Face Datasets used in Masked Face Recognition.
  • Figure 4: Samples from real masked face datasets used in Face Mask Recognition.
  • Figure 5: Samples of synthetic masked faces from benchmark datasets.
  • ...and 3 more figures