Colors Matter: AI-Driven Exploration of Human Feature Colors
Rama Alyoubi, Taif Alharbi, Albatul Alghamdi, Yara Alshehri, Elham Alghamdi
TL;DR
This work addresses the challenge of accurately classifying human feature colors (skin tone, hair color, iris color) and undertones under diverse lighting. It introduces an AI-driven, multi-stage pipeline that leverages face detection, region segmentation, and color-space analysis (LAB/HSV), combined with perceptual color differences ($\,Delta E_{00}$) and clustering (X-means) to map dominant colors to a custom skin-tone scale and undertone categories. Key contributions include a two-dataset evaluation setup, a custom 8-class skin tone scale, and demonstration that Delta E00 in HSV with Gaussian blur yields up to 0.80 accuracy for skin-tone classification, alongside robust methods for hair and iris color classification, and vein-based undertone assessment. The framework supports inclusive, precise color-based personalization with potential applications in beauty tech and digital identity, and the authors provide dataset access and code repositories to facilitate further research.
Abstract
This study presents a robust framework that leverages advanced imaging techniques and machine learning for feature extraction and classification of key human attributes-namely skin tone, hair color, iris color, and vein-based undertones. The system employs a multi-stage pipeline involving face detection, region segmentation, and dominant color extraction to isolate and analyze these features. Techniques such as X-means clustering, alongside perceptually uniform distance metrics like Delta E (CIEDE2000), are applied within both LAB and HSV color spaces to enhance the accuracy of color differentiation. For classification, the dominant tones of the skin, hair, and iris are extracted and matched to a custom tone scale, while vein analysis from wrist images enables undertone classification into "Warm" or "Cool" based on LAB differences. Each module uses targeted segmentation and color space transformations to ensure perceptual precision. The system achieves up to 80% accuracy in tone classification using the Delta E-HSV method with Gaussian blur, demonstrating reliable performance across varied lighting and image conditions. This work highlights the potential of AI-powered color analysis and feature extraction for delivering inclusive, precise, and nuanced classification, supporting applications in beauty technology, digital personalization, and visual analytics.