Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing 4D Lookup Table for Low-light Video Enhancement via Wavelet Priori

Jinhong He, Minglong Xue, Wenhai Wang, Mingliang Zhou

TL;DR

Wavelet-priori for 4D Lookup Table (WaveLUT) is proposed, which effectively enhances the color coherence between video frames and the accuracy of color mapping while maintaining low latency and achieves metric-favorable and perceptually oriented real-time enhancement while maintaining high efficiency.

Abstract

Low-light video enhancement is highly demanding in maintaining spatiotemporal color consistency. Therefore, improving the accuracy of color mapping and keeping the latency low is challenging. Based on this, we propose incorporating Wavelet-priori for 4D Lookup Table (WaveLUT), which effectively enhances the color coherence between video frames and the accuracy of color mapping while maintaining low latency. Specifically, we use the wavelet low-frequency domain to construct an optimized lookup prior and achieve an adaptive enhancement effect through a designed Wavelet-prior 4D lookup table. To effectively compensate the a priori loss in the low light region, we further explore a dynamic fusion strategy that adaptively determines the spatial weights based on the correlation between the wavelet lighting prior and the target intensity structure. In addition, during the training phase, we devise a text-driven appearance reconstruction method that dynamically balances brightness and content through multimodal semantics-driven Fourier spectra. Extensive experiments on a wide range of benchmark datasets show that this method effectively enhances the previous method's ability to perceive the color space and achieves metric-favorable and perceptually oriented real-time enhancement while maintaining high efficiency.

Optimizing 4D Lookup Table for Low-light Video Enhancement via Wavelet Priori

TL;DR

Wavelet-priori for 4D Lookup Table (WaveLUT) is proposed, which effectively enhances the color coherence between video frames and the accuracy of color mapping while maintaining low latency and achieves metric-favorable and perceptually oriented real-time enhancement while maintaining high efficiency.

Abstract

Low-light video enhancement is highly demanding in maintaining spatiotemporal color consistency. Therefore, improving the accuracy of color mapping and keeping the latency low is challenging. Based on this, we propose incorporating Wavelet-priori for 4D Lookup Table (WaveLUT), which effectively enhances the color coherence between video frames and the accuracy of color mapping while maintaining low latency. Specifically, we use the wavelet low-frequency domain to construct an optimized lookup prior and achieve an adaptive enhancement effect through a designed Wavelet-prior 4D lookup table. To effectively compensate the a priori loss in the low light region, we further explore a dynamic fusion strategy that adaptively determines the spatial weights based on the correlation between the wavelet lighting prior and the target intensity structure. In addition, during the training phase, we devise a text-driven appearance reconstruction method that dynamically balances brightness and content through multimodal semantics-driven Fourier spectra. Extensive experiments on a wide range of benchmark datasets show that this method effectively enhances the previous method's ability to perceive the color space and achieves metric-favorable and perceptually oriented real-time enhancement while maintaining high efficiency.
Paper Structure (25 sections, 16 equations, 10 figures, 5 tables)

This paper contains 25 sections, 16 equations, 10 figures, 5 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Low-light Image Enhancement
  4. Low-light Video Enhancement
  5. Lookup Table in Enhancement Tasks
  6. Method
  7. Lookup Tables Preliminaries
  8. 4D LUT Construction and Transformation
  9. Wavelet-prior 4D LUT
  10. The Interpolation Step
  11. Light-Enhanced 4D LUT Transformation
  12. Text-Driven Appearance Reconstruction
  13. Model Training
  14. EXPERIMENTS
  15. Experimental Settings
  16. ...and 10 more sections

Figures (10)

  • Figure 1: Comparison with FASTLLVE on the pixel distribution of the enhancement results. It can be seen that we mapped pixel values closer to the reference video.
  • Figure 2: Our method is compared with SOTA method DP3DF xu2023deep, FASTLLVEli2023fastllve. We can achieve more accurate color mapping, resulting in a friendlier visual effect.
  • Figure 3: Based on the SMID dataset, we use the Nvidia RTX 3090 GPU to target $1920\times1080$ (1080p) video comprehensively comparing PSNR and efficiency. Our approach has a better balance in the evaluation of performance and efficiency.
  • Figure 4: The overall workflow of our proposed WaveLUT. First it extracts the wavelet low-frequency domain of low-light video by discrete wavelet transform $(DWT)$. Subsequently, the wavelet low-frequency domain is used to guide the construction of the Wavelet-prior 4D LUT and optimize the lookup prior in combination with the dynamic fusion strategy. Based on the constructed Wavelet-prior 4D LUT and lookup prior, the input low-light video is converted to enhanced high-light video by light-enhanced 4D LUT transformation. Finally, during the training phase, we further optimize the enhancement results through text-driven appearance reconstruction.
  • Figure 5: Workflow of the dynamic fusion strategy. The lighting prior generated in the wavelet low-frequency domain and the intensity map generated from the low-light video are weighted and summed for similarity to generate an optimized lookup priori.
  • ...and 5 more figures