Controllable and Gradual Facial Blemishes Retouching via Physics-Based Modelling

TL;DR

This paper tackles the problem of realistic, gradual blemish retouching, aiming to visualize skincare efficacy rather than merely remove blemishes. It introduces CGFR, a physics-based approach that decomposes skin color into a Melanin–haemoglobin chromophore space and uses a Sum-of-Gaussians diffusion layer to model subsurface scattering, enabling per-blemish, chromophore-level control. The method fits a multi-Gaussian blemish model and applies a user-controlled gain to simulate fading, validated on clinical data and via a perception study that shows high realism and indistinguishability from real images. CGFR offers a data-light alternative to deep learning-based methods, with potential impact on the cosmetic industry for product visualization and skin science research.

Abstract

Face retouching aims to remove facial blemishes, such as pigmentation and acne, and still retain fine-grain texture details. Nevertheless, existing methods just remove the blemishes but focus little on realism of the intermediate process, limiting their use more to beautifying facial images on social media rather than being effective tools for simulating changes in facial pigmentation and ance. Motivated by this limitation, we propose our Controllable and Gradual Face Retouching (CGFR). Our CGFR is based on physical modelling, adopting Sum-of-Gaussians to approximate skin subsurface scattering in a decomposed melanin and haemoglobin color space. Our CGFR offers a user-friendly control over the facial blemishes, achieving realistic and gradual blemishes retouching. Experimental results based on actual clinical data shows that CGFR can realistically simulate the blemishes' gradual recovering process.

Figures (6)

• Figure 1: An overview of our CGFR pipeline. In our pipeline, a box of Region of Interest (ROI) is first used to select the blemish like pigmentation or acne. Then, a Layer Separation Filter is applied to separate the texture layer and the diffusion layers. A Sum-of-Gaussians model is fitted to each ROI in Melanin/haemoglobin color space, with the parameters of the fitted model adjusted to manipulate the appearance of the blemishes. The modified diffusion layer is summed with the original texture layer to obtain the output.
• Figure 2: Layered skin model. A portion of the incident light undergoes specular reflection, revealed as a skin texture layer. The other part transmits into and is scattered by the Epidermis and Dermis. Melanin and haemoglobin, which are distributed in these two layers, absorb specific wavelengths of light, rendering the skin's characteristic color.
• Figure 3: Matrix of different chromophore concentrations setting. We set different gains $\alpha$ according to equation \ref{['eqn:alpha']}, shown as percentage values in the figure. The original image is marked by a red box. Our model fully decouples the major chromophores of human skin, enabling highly controllable blemish editing.
• Figure 4: Comparison with baseline methods. We compared the results of several blemish removal or modification methods, including our method (marked as Ours), Adobe Photoshopadobephotoshop inpainting (marked as PS), and Stable Diffusionrombach2021highresolution inpainting (marked as SD). Arrows are manually added to highlight user-selected blemishes. Note the red arrows where the PS produces over-smoothed skin patches and the SD produces visible artifacts.
• Figure 5: Metadata and results of the perception study. The test population covers people from 19 to 45 years old, multiple races, and multiple skin tones, as shown in Fig.\ref{['fig:metadata']}. Panellists scored images from -2 to +2 to assess their confidence in considering the image as modified or not, with higher scores indicating that the user considered the image to be unmodified. Scoring frequencies are displayed in Fig.\ref{['fig:survey_hist']}. The results of the survey are shown in Fig.\ref{['fig:bar_charts']}. For the modified images, more people perceived them as unmodified or not sure. This suggests that our modifications are consistent with human perception and intuition.