Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Two-Step Data Augmentation for Masked Face Detection and Recognition: Turning Fake Masks to Real

Yan Yang, George Bebis, Mircea Nicolescu

TL;DR

Masked-face datasets are limited, hindering robust detection and recognition under occlusion. The authors propose a two-step augmentation: rule-based mask warping to generate guided inputs, followed by an unpaired image-to-image translation (AttentionGAN-like) that renders more realistic masks, with a non-mask-change loss and stochastic noise to stabilize training and boost diversity. The approach yields qualitative improvements over rule-based warping and complements IAMGAN's GAN-based generation, addressing both realism and coverage of mask variations. Limitations include potential overfitting on small datasets and limited mask-type diversity, pointing to future work on larger, more varied data and refined losses.

Abstract

Data scarcity and distribution shift pose major challenges for masked face detection and recognition. We propose a two-step generative data augmentation framework that combines rule-based mask warping with unpaired image-to-image translation using GANs, enabling the generation of realistic masked-face samples beyond purely synthetic transformations. Compared to rule-based warping alone, the proposed approach yields consistent qualitative improvements and complements existing GAN-based masked face generation methods such as IAMGAN. We introduce a non-mask preservation loss and stochastic noise injection to stabilize training and enhance sample diversity. Experimental observations highlight the effectiveness of the proposed components and suggest directions for future improvements in data-centric augmentation for face recognition tasks.

Two-Step Data Augmentation for Masked Face Detection and Recognition: Turning Fake Masks to Real

TL;DR

Masked-face datasets are limited, hindering robust detection and recognition under occlusion. The authors propose a two-step augmentation: rule-based mask warping to generate guided inputs, followed by an unpaired image-to-image translation (AttentionGAN-like) that renders more realistic masks, with a non-mask-change loss and stochastic noise to stabilize training and boost diversity. The approach yields qualitative improvements over rule-based warping and complements IAMGAN's GAN-based generation, addressing both realism and coverage of mask variations. Limitations include potential overfitting on small datasets and limited mask-type diversity, pointing to future work on larger, more varied data and refined losses.

Abstract

Data scarcity and distribution shift pose major challenges for masked face detection and recognition. We propose a two-step generative data augmentation framework that combines rule-based mask warping with unpaired image-to-image translation using GANs, enabling the generation of realistic masked-face samples beyond purely synthetic transformations. Compared to rule-based warping alone, the proposed approach yields consistent qualitative improvements and complements existing GAN-based masked face generation methods such as IAMGAN. We introduce a non-mask preservation loss and stochastic noise injection to stabilize training and enhance sample diversity. Experimental observations highlight the effectiveness of the proposed components and suggest directions for future improvements in data-centric augmentation for face recognition tasks.

Paper Structure

This paper contains 12 sections, 9 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. DATA
  4. METHOD
  5. Adding Non-Mask Change Loss
  6. Adding Noise
  7. Transfer Learning and the Training Timeline
  8. DISCUSSIONS
  9. Improvements on Top of CMFD
  10. Comparison with IAMGAN
  11. Potential Improvements
  12. CONCLUSIONS

Figures (9)

  • Figure 1: Examples from dataset B.
  • Figure 2: Generator architecture in AttentionGAN. Our noise inputs are depicted in red.
  • Figure 3: Feed noise to the last two content-generating layers without training.
  • Figure 4: Uniform-colored masks generated at epoch 279.
  • Figure 5: Models drastically distort inputs at two different epochs. Outputs like images 2 and 4 intermittently appear as the training progresses.
  • ...and 4 more figures