Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lookism: The overlooked bias in computer vision

Aditya Gulati, Bruno Lepri, Nuria Oliver

TL;DR

The paper tackles lookism, a bias based on physical appearance, as an under-explored risk in computer vision. It surveys how lookism can arise both in user-facing beauty filters and in algorithmic systems, illustrating with a large crowdsourced study and prompts-based generation examples. The authors highlight three intersecting areas between lookism and CV and advocate for interdisciplinary work, diverse data, fairness-aware designs, and continuous auditing to build equitable CV that respects human appearance diversity. The findings underscore the potential for lookism to reinforce stereotypes and unequal treatment, urging researchers, policymakers, and developers to prioritize mitigation.

Abstract

In recent years, there have been significant advancements in computer vision which have led to the widespread deployment of image recognition and generation systems in socially relevant applications, from hiring to security screening. However, the prevalence of biases within these systems has raised significant ethical and social concerns. The most extensively studied biases in this context are related to gender, race and age. Yet, other biases are equally pervasive and harmful, such as lookism, i.e., the preferential treatment of individuals based on their physical appearance. Lookism remains under-explored in computer vision but can have profound implications not only by perpetuating harmful societal stereotypes but also by undermining the fairness and inclusivity of AI technologies. Thus, this paper advocates for the systematic study of lookism as a critical bias in computer vision models. Through a comprehensive review of existing literature, we identify three areas of intersection between lookism and computer vision. We illustrate them by means of examples and a user study. We call for an interdisciplinary approach to address lookism, urging researchers, developers, and policymakers to prioritize the development of equitable computer vision systems that respect and reflect the diversity of human appearances.

Lookism: The overlooked bias in computer vision

TL;DR

The paper tackles lookism, a bias based on physical appearance, as an under-explored risk in computer vision. It surveys how lookism can arise both in user-facing beauty filters and in algorithmic systems, illustrating with a large crowdsourced study and prompts-based generation examples. The authors highlight three intersecting areas between lookism and CV and advocate for interdisciplinary work, diverse data, fairness-aware designs, and continuous auditing to build equitable CV that respects human appearance diversity. The findings underscore the potential for lookism to reinforce stereotypes and unequal treatment, urging researchers, policymakers, and developers to prioritize mitigation.

Abstract

In recent years, there have been significant advancements in computer vision which have led to the widespread deployment of image recognition and generation systems in socially relevant applications, from hiring to security screening. However, the prevalence of biases within these systems has raised significant ethical and social concerns. The most extensively studied biases in this context are related to gender, race and age. Yet, other biases are equally pervasive and harmful, such as lookism, i.e., the preferential treatment of individuals based on their physical appearance. Lookism remains under-explored in computer vision but can have profound implications not only by perpetuating harmful societal stereotypes but also by undermining the fairness and inclusivity of AI technologies. Thus, this paper advocates for the systematic study of lookism as a critical bias in computer vision models. Through a comprehensive review of existing literature, we identify three areas of intersection between lookism and computer vision. We illustrate them by means of examples and a user study. We call for an interdisciplinary approach to address lookism, urging researchers, developers, and policymakers to prioritize the development of equitable computer vision systems that respect and reflect the diversity of human appearances.
Paper Structure (10 sections, 2 figures)

This paper contains 10 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Lookism: A cognitive bias perspective
  3. Lookism and computer vision
  4. Beauty Filters and Lookism
  5. Algorithmic Lookism
  6. Lookism in Image Generation
  7. Lookism in Decision-making Systems
  8. Challenges
  9. Conclusion
  10. Acknowledgments

Figures (2)

  • Figure 1: An example of images without (left) and with (right) beauty filters applied from our datasets.
  • Figure 2: Illustrative examples of lookism in T2I models. Images generated by DALL-E3 through Microsoft's Copilot with prompts: "create a hyperrealistic portrait of (a) an intelligent person; (b) an unintelligent person; (c) a very competent person; and (d) a very incompetent person".