Deep chroma compression of tone-mapped images

TL;DR

Quantitative experiments demonstrate that the proposed model outperforms state-of-the-art image generation and enhancement networks in color accuracy, while a subjective study suggests that the generated images are on par or superior to those produced by conventional chroma compression methods in terms of visual quality.

Abstract

Acquisition of high dynamic range (HDR) images is thriving due to the increasing use of smart devices and the demand for high-quality output. Extensive research has focused on developing methods for reducing the luminance range in HDR images using conventional and deep learning-based tone mapping operators to enable accurate reproduction on conventional 8 and 10-bit digital displays. However, these methods often fail to account for pixels that may lie outside the target display's gamut, resulting in visible chromatic distortions or color clipping artifacts. Previous studies suggested that a gamut management step ensures that all pixels remain within the target gamut. However, such approaches are computationally expensive and cannot be deployed on devices with limited computational resources. We propose a generative adversarial network for fast and reliable chroma compression of HDR tone-mapped images. We design a loss function that considers the hue property of generated images to improve color accuracy, and train the model on an extensive image dataset. Quantitative experiments demonstrate that the proposed model outperforms state-of-the-art image generation and enhancement networks in color accuracy, while a subjective study suggests that the generated images are on par or superior to those produced by conventional chroma compression methods in terms of visual quality. Additionally, the model achieves real-time performance, showing promising results for deployment on devices with limited computational resources.

Figures (8)

• Figure 1: Accurate and efficient chroma compression is important for the reproduction of HDR images on standard displays with limited computational resources. This study proposes a deep learning-based method for chroma compression that is up to 3 orders of magnitude faster than the conventional approach, while generating pixel values that similarly fall within the target display's gamut (here $sRGB$). The scatter plot shows the chroma values ($x$ axis) of pixels within a hue slice ($h=60^{\circ}$) against their lightness values ($y$ axis). Tone mapping cannot compress all pixels within the gamut boundaries, a problem that chroma compression is able to solve.
• Figure 2: Overview of method and evaluation. HDR images are tone-mapped and then chroma-compressed into SDR images. Our generative adversarial network learns the same process, using a U-Net generator, $G$, and a pixel-based discriminator, $D$. The final images predicted by the trained network are compared against the reference chroma-compressed images for evaluation.
• Figure 3: (a) Selected images for the subjective evaluation. (b) The experimental setup in a room with dim lights on for visualization purposes.
• Figure 4: An example image tone-mapped (dashed frame) and subsequently chroma-compressed by the reference method, various SOTA models for image restoration, and our method. Corresponding CVVDP distortion maps are also shown (blue indicates smaller distortion and red/pink indicates larger distortion).
• Figure 5: The effect of the size of the discriminator's receptive field. The 70$\times$70 patch-based discriminator occasionally generates tiling artifacts that the pixel-based discriminator can suppress. The original tone-mapped image has a dashed frame.