Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Working with Color: How Color Quantization Can Aid Researchers of Problematic Information

Nina Lutz, Jordyn W. Padzensky, Joseph S. Schafer

TL;DR

This paper addresses the challenge of analyzing large image collections of problematic information without over-relying on AI due to biases and computational costs. It introduces a lightweight, non-AI color quantization method embedded in a mixed-methods, human-in-the-loop pipeline to study visual rhetoric around the US–Mexico border. The approach leverages HSV-based color quantization, k-means clustering, Monk Skin Tone orb palettes, and symbol palettes, integrated with Label Studio for qualitative coding and sample selection, with a media-forensics use case. The study discusses limitations and advocates a historicist, critically reflective reclamation of such methods for scalable, ethical visual research in CSCW and misinformation studies.

Abstract

Analyzing large sets of visual media remains a challenging task, particularly in mixed-method studies dealing with problematic information and human subjects. Using AI tools in such analyses risks reifying and exacerbating biases, as well as untenable computational and cost limitations. As such, we turn to adopting geometric computer graphics and vision methods towards analyzing a large set of images from a problematic information campaign, in conjunction with human-in-the-loop qualitative analysis. We illustrate an effective case of this approach with the implementation of color quantization towards analyzing online hate image at the US-Mexico border, along with a historicist trace of the history of color quantization and skin tone scales, to inform our usage and reclamation of these methodologies from their racist origins. To that end, we scaffold motivations and the need for more researchers to consider the advantages and risks of reclaiming such methodologies in their own work, situated in our case study.

Working with Color: How Color Quantization Can Aid Researchers of Problematic Information

TL;DR

This paper addresses the challenge of analyzing large image collections of problematic information without over-relying on AI due to biases and computational costs. It introduces a lightweight, non-AI color quantization method embedded in a mixed-methods, human-in-the-loop pipeline to study visual rhetoric around the US–Mexico border. The approach leverages HSV-based color quantization, k-means clustering, Monk Skin Tone orb palettes, and symbol palettes, integrated with Label Studio for qualitative coding and sample selection, with a media-forensics use case. The study discusses limitations and advocates a historicist, critically reflective reclamation of such methods for scalable, ethical visual research in CSCW and misinformation studies.

Abstract

Analyzing large sets of visual media remains a challenging task, particularly in mixed-method studies dealing with problematic information and human subjects. Using AI tools in such analyses risks reifying and exacerbating biases, as well as untenable computational and cost limitations. As such, we turn to adopting geometric computer graphics and vision methods towards analyzing a large set of images from a problematic information campaign, in conjunction with human-in-the-loop qualitative analysis. We illustrate an effective case of this approach with the implementation of color quantization towards analyzing online hate image at the US-Mexico border, along with a historicist trace of the history of color quantization and skin tone scales, to inform our usage and reclamation of these methodologies from their racist origins. To that end, we scaffold motivations and the need for more researchers to consider the advantages and risks of reclaiming such methodologies in their own work, situated in our case study.
Paper Structure (17 sections, 7 figures)

This paper contains 17 sections, 7 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Visual Misinformation
  4. Historicism in CSCW and Color Quantization
  5. Digital Color Quantization
  6. Methods
  7. Color Quantization Implementation
  8. Human Subject or Chosen Object Implementation
  9. Mixed-Methods Human-in-the-Loop Content Analysis
  10. Results
  11. Human and Select Subjects of Interest
  12. Human-in-the-loop Interaction
  13. Media Forensics
  14. Discussion
  15. Limitations
  16. ...and 2 more sections

Figures (7)

  • Figure 1: Showing color spaces in RGB and HSV geometries. Figure credit: Michael Horvath wiki.
  • Figure 2: Differences between HSV and RGB spaces in image color quantization. While the summative palette remains the same, the left HSV space distinguishes the hues for more intuitive human summarization. Source; Lutz, 2020 lutz_cuties_2020.
  • Figure 3: On the left, the pixel color distribution of the image on the right in HSV color space, with centroids plotted as larger points. The summative color palette for the image is below the image itself. This image was selected from our sample set of imagery on misinformation and online hate about the US-Mexico border rhetoric.
  • Figure 4: Monk scale and orb palettes. We use starting HEX colors and projected them into HSV value ranges with 15% bound on all axes permutations, to capture colors across the orb distribution and within our images. Figure credit: Dr. Ellis Monk monk_monk_2023.
  • Figure 5: Summative palette extracted from Biden image, with the green highlighted color falling into the Monk Skin Tone Spectrum (below palette). Face blurred for privacy schafer_screenshot_2024. This image was Photoshopped and is not an accurate image, and has been used as part of an online hate campaign against Latin American migrants.
  • ...and 2 more figures