A Color Image Analysis Tool to Help Users Choose a Makeup Foundation Color

TL;DR

We address the problem of predicting skin-with-foundation color from a no-makeup selfie and foundation swatch by building a calibrated color pipeline and learning a color-difference regression. The method calibrates device RGB to standard color space using three transformation matrices $T_1$, $T_2$, $T_3$ learned from color-checker patches partitioned into three sets, then converts colors to the CIE $L^*a^*b^*$ space for skin, foundation, and skin-with-foundation regions. Two regression approaches, linear regression and SVR with a linear kernel, are trained to predict the skin-with-foundation $L^*a^*b^*$ from the inputs, with leave-one-out cross-validation yielding $R^2$ values around $0.83$ and $0.82$ respectively and MSE/MAE around $1.49$–$1.50$ and $0.87$–$0.91$, respectively. Calibration achieves near-invisible color differences with $igtriangleup E_{76}$ typically $\lex 1$ for most patches, supporting accurate foundation shade prediction and practical makeup color matching.

Abstract

This paper presents an approach to predict the color of skin-with-foundation based on a no makeup selfie image and a foundation shade image. Our approach first calibrates the image with the help of the color checker target, and then trains a supervised-learning model to predict the skin color. In the calibration stage, We propose to use three different transformation matrices to map the device dependent RGB response to the reference CIE XYZ space. In so doing, color correction error can be minimized. We then compute the average value of the region of interest in the calibrated images, and feed them to the prediction model. We explored both the linear regression and support vector regression models. Cross-validation results show that both models can accurately make the prediction.