NCST: Neural-based Color Style Transfer for Video Retouching

TL;DR

A method that predicts specific parameters for color style transfer using two images, and each parameter has a specific, interpretable meaning, enabling users to understand the color style transfer process and allowing them to perform manual fine-tuning if desired.

Abstract

Video color style transfer aims to transform the color style of an original video by using a reference style image. Most existing methods employ neural networks, which come with challenges like opaque transfer processes and limited user control over the outcomes. Typically, users cannot fine-tune the resulting images or videos. To tackle this issue, we introduce a method that predicts specific parameters for color style transfer using two images. Initially, we train a neural network to learn the corresponding color adjustment parameters. When applying style transfer to a video, we fine-tune the network with key frames from the video and the chosen style image, generating precise transformation parameters. These are then applied to convert the color style of both images and videos. Our experimental results demonstrate that our algorithm surpasses current methods in color style transfer quality. Moreover, each parameter in our method has a specific, interpretable meaning, enabling users to understand the color style transfer process and allowing them to perform manual fine-tuning if desired.

Figures (8)

• Figure 1: Color Style Transfer. Given a content image and a style image, our method is able to transfer the color style from the style image to the content image with the color parameters.
• Figure 2: An overview of neural-based color style transfer. Our method consists of three steps: (a). Pre-Training, images are randomly selected from a large image dataset as style images and content images for pretraining to obtain a pretrained model. (b). Test-On-Time Training, the user's specific content video and style image are input into this pretrained model for fine-tuning, resulting in a specific color style transfer model used to generate parameters like brightness and contrast, forming a parameter set. (c). Retorch Videos, this parameter set is used to perform color style transfer on the content video, resulting in a stylized video.
• Figure 3: Visual comparison of our method with other methods on multiple images. These images can show our methods has better result compared with other methods
• Figure 4: The impact of various color grading parameters on the final color grading effect. This experiment demonstrates the impact of removing individual adjustment parameters on style transfer results. The results show that each parameter contributes differently to the final effect.
• Figure 5: The impact of converting color grading parameters into 3D LUT on color grading effects. In this experiment, we predicted color grading parameters while generating a 3D LUT and applied it in DaVinci Resolve. The "Content" refers to the content images, "Style" refers to the style images, "Normal" refers to results obtained directly using our method, and "Combine LUT" shows the effects of applying the generated LUT in DaVinci Resolve.