Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Facial Attribute Based Text Guided Face Anonymization

Mustafa İzzet Muştu, Hazım Kemal Ekenel

TL;DR

A deep learning-based face anonymization pipeline to overcome the challenge of data privacy regulations, and leverages recent advancements in diffusion-based inpainting models, eliminating the need for training Generative Adversarial Networks.

Abstract

The increasing prevalence of computer vision applications necessitates handling vast amounts of visual data, often containing personal information. While this technology offers significant benefits, it should not compromise privacy. Data privacy regulations emphasize the need for individual consent for processing personal data, hindering researchers' ability to collect high-quality datasets containing the faces of the individuals. This paper presents a deep learning-based face anonymization pipeline to overcome this challenge. Unlike most of the existing methods, our method leverages recent advancements in diffusion-based inpainting models, eliminating the need for training Generative Adversarial Networks. The pipeline employs a three-stage approach: face detection with RetinaNet, feature extraction with VGG-Face, and realistic face generation using the state-of-the-art BrushNet diffusion model. BrushNet utilizes the entire image, face masks, and text prompts specifying desired facial attributes like age, ethnicity, gender, and expression. This enables the generation of natural-looking images with unrecognizable individuals, facilitating the creation of privacy-compliant datasets for computer vision research.

Facial Attribute Based Text Guided Face Anonymization

TL;DR

A deep learning-based face anonymization pipeline to overcome the challenge of data privacy regulations, and leverages recent advancements in diffusion-based inpainting models, eliminating the need for training Generative Adversarial Networks.

Abstract

The increasing prevalence of computer vision applications necessitates handling vast amounts of visual data, often containing personal information. While this technology offers significant benefits, it should not compromise privacy. Data privacy regulations emphasize the need for individual consent for processing personal data, hindering researchers' ability to collect high-quality datasets containing the faces of the individuals. This paper presents a deep learning-based face anonymization pipeline to overcome this challenge. Unlike most of the existing methods, our method leverages recent advancements in diffusion-based inpainting models, eliminating the need for training Generative Adversarial Networks. The pipeline employs a three-stage approach: face detection with RetinaNet, feature extraction with VGG-Face, and realistic face generation using the state-of-the-art BrushNet diffusion model. BrushNet utilizes the entire image, face masks, and text prompts specifying desired facial attributes like age, ethnicity, gender, and expression. This enables the generation of natural-looking images with unrecognizable individuals, facilitating the creation of privacy-compliant datasets for computer vision research.

Paper Structure

This paper contains 6 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Experiments
  5. Conclusions and Future Work
  6. Acknowledgements

Figures (3)

  • Figure 1: Proposed pipeline. Face detection is done by using RetinaFace. Facial attribute analysis is performed by VGG-Face and then these attributes are converted to text. Finally, faces are generated by BrushNet
  • Figure 2: Qualitative results. Example face anonymization results. The first row is the original images, the second row is the results of LDFA and the last row is results of our pipeline
  • Figure 3: Limitations. The top row is the original images and the bottom row is generated images. The first column is related to the capabilities of the inpainting model and the second one is mostly related to the resolution of the facial region