NamedCurves: Learned Image Enhancement via Color Naming

TL;DR

This work proposes NamedCurves, a learning-based image enhancement technique that separates the image into a small set of named colors via tone curves and then combines the images using an attention-based fusion mechanism to mimic spatial editing.

Abstract

A popular method for enhancing images involves learning the style of a professional photo editor using pairs of training images comprised of the original input with the editor-enhanced version. When manipulating images, many editing tools offer a feature that allows the user to manipulate a limited selection of familiar colors. Editing by color name allows easy adjustment of elements like the "blue" of the sky or the "green" of trees. Inspired by this approach to color manipulation, we propose NamedCurves, a learning-based image enhancement technique that separates the image into a small set of named colors. Our method learns to globally adjust the image for each specific named color via tone curves and then combines the images using an attention-based fusion mechanism to mimic spatial editing. We demonstrate the effectiveness of our method against several competing methods on the well-known Adobe 5K dataset and the PPR10K dataset, showing notable improvements.

Figures (14)

• Figure 1: Column 1 displays an input image corrected by a photo-editing expert (denoted as ground truth). Our proposed method decomposes the image based on color naming and learns a tone-curve correction to mimic the expert's style (shown in columns 2-3). Results comparing the input, our results, and the approach by moran2020deeplpf are reported in terms of the color distance metric $\Delta E_{00}$.
• Figure 2: Van de Weijer et al. van2009learning color names grouped in the Munsell color array. The color names are orange, brown, yellow, white, grey, black, pink, purple, red, green and blue.
• Figure 3: Van de Weijer et al. van2009learning color naming method applied pixel-wise to the top-left image. The other 11 images show the 11 probability color names maps. Each color is displayed with a different map to aid visualization. Note that some linguistic color names share approximately the same hue and only differ in intensity--- e.g., pink and purple. As tone curves are defined for all the intensity range we group: orange-brown-yellow, pink-purple, and white-grey-black. This grouping is represented by the boxes.
• Figure 4: Overview of the proposed method, NamedCurves. Our method aims to enhance an input image $x$. First, we use a UNet-like backbone to standardize the input image into a canonical latent space. Next, we decompose the standardized image $\hat{y}_b$ into six color probability maps (shown color-coded in the figure to aid visualization). Next, we learn a set of Bezier curves for each color name to manipulate the standardized image $\hat{y}_b$, obtaining six distinct globally adjusted images. Finally, an attention mechanism is used to fuse the edited images using as Query the standardized image $\hat{y}_b$ and as Key and Value the corresponding edited image. Our learning-based method uses an objective function $L$ to compare the expert-retouched image with our final result $\hat{y}$.
• Figure 5: Bezier Control Point Estimator (BCPE). First, we extract 64-D convolutional feature maps from $\hat{y}_b$ using mainly 4 Conv-ReLU blocks. Then, the color naming is concatenated and passed through 4 Conv-ReLU-MaxPooling and a final AveragePooling layer. The output of the module is $\Delta P$, the unnormalized control points increments.