Quality-guided Skin Tone Enhancement for Portrait Photography

TL;DR

A quality-guided image enhancement paradigm is proposed that enables image enhancement models to learn the distribution of images with various quality ratings, which can be used to adjust images continuously according to different quality scores.

Abstract

In recent years, learning-based color and tone enhancement methods for photos have become increasingly popular. However, most learning-based image enhancement methods just learn a mapping from one distribution to another based on one dataset, lacking the ability to adjust images continuously and controllably. It is important to enable the learning-based enhancement models to adjust an image continuously, since in many cases we may want to get a slighter or stronger enhancement effect rather than one fixed adjusted result. In this paper, we propose a quality-guided image enhancement paradigm that enables image enhancement models to learn the distribution of images with various quality ratings. By learning this distribution, image enhancement models can associate image features with their corresponding perceptual qualities, which can be used to adjust images continuously according to different quality scores. To validate the effectiveness of our proposed method, a subjective quality assessment experiment is first conducted, focusing on skin tone adjustment in portrait photography. Guided by the subjective quality ratings obtained from this experiment, our method can adjust the skin tone corresponding to different quality requirements. Furthermore, an experiment conducted on 10 natural raw images corroborates the effectiveness of our model in situations with fewer subjects and fewer shots, and also demonstrates its general applicability to natural images. Our project page is https://github.com/IntMeGroup/quality-guided-enhancement .

Figures (18)

• Figure 1: Quality-guided Image Enhancement Paradigm. Our framework first applies image quality assessment for a raw image and various adjusted images to obtain the quality scores for the adjusted images, then utilizes a raw image and the corresponding quality score of an adjusted image as the input for the enhancement network, and optimizes the network to make the output enhanced image as close as possible to the adjusted image. "Positive" and "Negative" represent better perceptual quality and worse perceptual quality compared to the raw image, respectively.
• Figure 2: An illustration of the user interface of the subjective skin tone quality assessment experiment.
• Figure 3: Distribution of subjective quality ratings.
• Figure 4: An overview of the framework of our method. The image quality assessment and category labeling modules are shown in the orange dotted box. The image enhancement modules are shown in the blue dotted box.
• Figure 5: Skin tone image classification results based on Google monk skin tone.