Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hide and Seek: How Does Watermarking Impact Face Recognition?

Yuguang Yao, Steven Grosz, Sijia Liu, Anil Jain

TL;DR

This work investigates the impact of digital watermarking, a technique for embedding ownership signatures into images, on the effectiveness of face recognition models and proposes a comprehensive pipeline that integrates face image generation, watermarking, and face recognition to systematically examine this question.

Abstract

The recent progress in generative models has revolutionized the synthesis of highly realistic images, including face images. This technological development has undoubtedly helped face recognition, such as training data augmentation for higher recognition accuracy and data privacy. However, it has also introduced novel challenges concerning the responsible use and proper attribution of computer generated images. We investigate the impact of digital watermarking, a technique for embedding ownership signatures into images, on the effectiveness of face recognition models. We propose a comprehensive pipeline that integrates face image generation, watermarking, and face recognition to systematically examine this question. The proposed watermarking scheme, based on an encoder-decoder architecture, successfully embeds and recovers signatures from both real and synthetic face images while preserving their visual fidelity. Through extensive experiments, we unveil that while watermarking enables robust image attribution, it results in a slight decline in face recognition accuracy, particularly evident for face images with challenging poses and expressions. Additionally, we find that directly training face recognition models on watermarked images offers only a limited alleviation of this performance decline. Our findings underscore the intricate trade off between watermarking and face recognition accuracy. This work represents a pivotal step towards the responsible utilization of generative models in face recognition and serves to initiate discussions regarding the broader implications of watermarking in biometrics.

Hide and Seek: How Does Watermarking Impact Face Recognition?

TL;DR

This work investigates the impact of digital watermarking, a technique for embedding ownership signatures into images, on the effectiveness of face recognition models and proposes a comprehensive pipeline that integrates face image generation, watermarking, and face recognition to systematically examine this question.

Abstract

The recent progress in generative models has revolutionized the synthesis of highly realistic images, including face images. This technological development has undoubtedly helped face recognition, such as training data augmentation for higher recognition accuracy and data privacy. However, it has also introduced novel challenges concerning the responsible use and proper attribution of computer generated images. We investigate the impact of digital watermarking, a technique for embedding ownership signatures into images, on the effectiveness of face recognition models. We propose a comprehensive pipeline that integrates face image generation, watermarking, and face recognition to systematically examine this question. The proposed watermarking scheme, based on an encoder-decoder architecture, successfully embeds and recovers signatures from both real and synthetic face images while preserving their visual fidelity. Through extensive experiments, we unveil that while watermarking enables robust image attribution, it results in a slight decline in face recognition accuracy, particularly evident for face images with challenging poses and expressions. Additionally, we find that directly training face recognition models on watermarked images offers only a limited alleviation of this performance decline. Our findings underscore the intricate trade off between watermarking and face recognition accuracy. This work represents a pivotal step towards the responsible utilization of generative models in face recognition and serves to initiate discussions regarding the broader implications of watermarking in biometrics.
Paper Structure (18 sections, 3 equations, 9 figures, 4 tables)

This paper contains 18 sections, 3 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Face image generation.
  5. Digital watermarking.
  6. Face recognition.
  7. Methodology
  8. Digital Watermarking
  9. Integration of Watermark into Face Images
  10. Face Recognition on Watermarked Images
  11. Experiments
  12. Experimental Setup
  13. Experimental Results
  14. Assessing watermarking quality in face images.
  15. Watermarking effect on pre-trained face recognition models.
  16. ...and 3 more sections

Figures (9)

  • Figure 1: The schematic overview of the studied face recognition pipeline, which consists of face image generation (top), digital watermarking (middle), and face recognition (bottom). The watermarking technique follows HiDDeN zhu2018hidden.
  • Figure 2: The schematic overview of watermarking system.
  • Figure 3: The embedded watermarking signature.
  • Figure 4: Visualizations of watermarked images and their decoded watermark message (the '$S$' letter) based on the real image source CASIA-WebFace and the synthetic image source DCFace. Each column represents a data transformation operation applied after image watermarking. 'Original' denotes no data transformation. 'Crop 80%' refers to randomly cropping the image to 80% of its original size. 'Resize 80%' refers to resizing the image to 80% of its original size. 'Bright 3x' refers to increasing the image brightness by a factor of 3. 'Contrast 3x' refers to increasing the image contrast by a factor of 3. 'JPEG 90%' refers to applying JPEG compression with a quality factor of 90%.
  • Figure 5: Visualization of original images, watermarked images, and their 10$\times$ pixel differences for both real and synthetic face image datasets, CASIA-WebFace yi2014learning and DCFace kim2023dcface, respectively. The pixel-wise difference is color-inverted for enhanced visualization, where white areas denote zero values.
  • ...and 4 more figures