Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Inclusive Face Recognition Through Synthetic Ethnicity Alteration

Praveen Kumar Chandaliya, Kiran Raja, Raghavendra Ramachandra, Zahid Akhtar, Christoph Busch

TL;DR

Face recognition systems exhibit systematic biases across ethnic groups due to unbalanced training data. The authors propose synthetic ethnicity alteration using GAN-based image-to-image translation to generate ethnicity-representative faces, constructing a balanced ETAT dataset and evaluating skin-tone realism with ITA, FIQA with EDC, and cross-ethnicity verification across four FR systems. Key contributions include the ETAT dataset (45,000 synthetic images across Asian, Black, and Indian), a framework to assess skin-tone fidelity via ITA, and a cross-ethnicity verification study that informs fairness considerations and potential privacy benefits. The findings suggest synthetic ethnicity alteration can support fairness research and data expansion while offering privacy advantages, with future work pointing to diffusion-based methods for richer and more robust ethnicity representations.

Abstract

Numerous studies have shown that existing Face Recognition Systems (FRS), including commercial ones, often exhibit biases toward certain ethnicities due to under-represented data. In this work, we explore ethnicity alteration and skin tone modification using synthetic face image generation methods to increase the diversity of datasets. We conduct a detailed analysis by first constructing a balanced face image dataset representing three ethnicities: Asian, Black, and Indian. We then make use of existing Generative Adversarial Network-based (GAN) image-to-image translation and manifold learning models to alter the ethnicity from one to another. A systematic analysis is further conducted to assess the suitability of such datasets for FRS by studying the realistic skin-tone representation using Individual Typology Angle (ITA). Further, we also analyze the quality characteristics using existing Face image quality assessment (FIQA) approaches. We then provide a holistic FRS performance analysis using four different systems. Our findings pave the way for future research works in (i) developing both specific ethnicity and general (any to any) ethnicity alteration models, (ii) expanding such approaches to create databases with diverse skin tones, (iii) creating datasets representing various ethnicities which further can help in mitigating bias while addressing privacy concerns.

Towards Inclusive Face Recognition Through Synthetic Ethnicity Alteration

TL;DR

Face recognition systems exhibit systematic biases across ethnic groups due to unbalanced training data. The authors propose synthetic ethnicity alteration using GAN-based image-to-image translation to generate ethnicity-representative faces, constructing a balanced ETAT dataset and evaluating skin-tone realism with ITA, FIQA with EDC, and cross-ethnicity verification across four FR systems. Key contributions include the ETAT dataset (45,000 synthetic images across Asian, Black, and Indian), a framework to assess skin-tone fidelity via ITA, and a cross-ethnicity verification study that informs fairness considerations and potential privacy benefits. The findings suggest synthetic ethnicity alteration can support fairness research and data expansion while offering privacy advantages, with future work pointing to diffusion-based methods for richer and more robust ethnicity representations.

Abstract

Numerous studies have shown that existing Face Recognition Systems (FRS), including commercial ones, often exhibit biases toward certain ethnicities due to under-represented data. In this work, we explore ethnicity alteration and skin tone modification using synthetic face image generation methods to increase the diversity of datasets. We conduct a detailed analysis by first constructing a balanced face image dataset representing three ethnicities: Asian, Black, and Indian. We then make use of existing Generative Adversarial Network-based (GAN) image-to-image translation and manifold learning models to alter the ethnicity from one to another. A systematic analysis is further conducted to assess the suitability of such datasets for FRS by studying the realistic skin-tone representation using Individual Typology Angle (ITA). Further, we also analyze the quality characteristics using existing Face image quality assessment (FIQA) approaches. We then provide a holistic FRS performance analysis using four different systems. Our findings pave the way for future research works in (i) developing both specific ethnicity and general (any to any) ethnicity alteration models, (ii) expanding such approaches to create databases with diverse skin tones, (iii) creating datasets representing various ethnicities which further can help in mitigating bias while addressing privacy concerns.
Paper Structure (19 sections, 9 figures, 1 table)

This paper contains 19 sections, 9 figures, 1 table.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Demographic differentials in face recognition
  4. Image-to-Image Translation in Face Alteration
  5. Investigation Methodology
  6. Dataset
  7. Ethnicity Alteration Training (ETAT) Dataset
  8. Racial Faces in-the Wild
  9. Qualitative Evaluation
  10. Asian $\mapsto$ Black, Indian
  11. Black $\mapsto$ Asian, Indian
  12. Indian $\mapsto$ Asian, Black
  13. Quantitative Evaluation
  14. Skin Tone Estimation
  15. Face image quality assessment
  16. ...and 4 more sections

Figures (9)

  • Figure 1: Our framework for generating multi-ethnicity data for biased performance for different ethnicities as compared to existing works in FRS.
  • Figure 2: Michael Jackson's facial appearance change due to cosmetic surgeries and skin disorder disease resulting in changes in ethnical appearance.
  • Figure 3: Ethnicity alteration results on Asian faces.
  • Figure 4: Ethnicity alteration results on Black faces.
  • Figure 5: Ethnicity alteration results on Indian faces.
  • ...and 4 more figures