Make Lossy Compression Meaningful for Low-Light Images

TL;DR

Experimental results show that the proposed joint solution achieves a significant improvement over different combinations of existing state-of-the-art sequential ``Compress before Enhance'' or ``Enhance before Compress'' solutions for low-light images, which would make lossy low-light image compression more meaningful.

Abstract

Low-light images frequently occur due to unavoidable environmental influences or technical limitations, such as insufficient lighting or limited exposure time. To achieve better visibility for visual perception, low-light image enhancement is usually adopted. Besides, lossy image compression is vital for meeting the requirements of storage and transmission in computer vision applications. To touch the above two practical demands, current solutions can be categorized into two sequential manners: ``Compress before Enhance (CbE)'' or ``Enhance before Compress (EbC)''. However, both of them are not suitable since: (1) Error accumulation in the individual models plagues sequential solutions. Especially, once low-light images are compressed by existing general lossy image compression approaches, useful information (e.g., texture details) would be lost resulting in a dramatic performance decrease in low-light image enhancement. (2) Due to the intermediate process, the sequential solution introduces an additional burden resulting in low efficiency. We propose a novel joint solution to simultaneously achieve a high compression rate and good enhancement performance for low-light images with much lower computational cost and fewer model parameters. We design an end-to-end trainable architecture, which includes the main enhancement branch and the signal-to-noise ratio (SNR) aware branch. Experimental results show that our proposed joint solution achieves a significant improvement over different combinations of existing state-of-the-art sequential ``Compress before Enhance'' or ``Enhance before Compress'' solutions for low-light images, which would make lossy low-light image compression more meaningful. The project is publicly available at: https://github.com/CaiShilv/Joint-IC-LL.

Figures (21)

• Figure 1: Compared with sequential solutions ("Compress before Enhance (CbE)" and "Enhance before Compress (EbC)"), our proposed joint solution has significantly greater advantages in terms of PSNR, MS-SSIM, and computational cost with even lower bits per pixel (BPP). As shown, our joint solution makes lossy low-light image compression meaningful with much better visibility for visual perception. In this teaser figure, the compression and low-light enhancement methods of sequential solutions are Cheng cheng2020learned and Xu2022 xu2022snr respectively. The example images in the figure are from the SID dataset chen2018learning. For more comparison qualitative results, please refer to the supplementary material.
• Figure 2: The network architecture of our joint solution of low-light image compression and enhancement. The left half of the figure contains two branches, the "Main Enhancement Branch" and the "SNR Aware Branch". The low-light image is fed into the "Main Enhancement Branch" to obtain the two-level enhanced compressed domain features ($y_{0}$/$y$) via "Feature Adaptive" modules ($f_{a0}$/$f_{a1}$). The "SNR Aware Branch" obtains local/non-local information by the SNR-map $s$ and compressed domain features ($y_0^{\prime}$/$y_{1}^{\prime}$). The right half of the figure contains the main decoder, entropy models, context model, and hyper encoder/decoder commonly used in recent learning-based compression methods minnen2018jointcheng2020learned. "/" means "or" in this paper.
• Figure 3: Architecture details of the "Feature Adaptive" module. SNR-aware fusion features ($s_{0}$/$s_{1}$) act as a condition on the compressed domain features ($y_0^{\prime}$/$y_{1}^{\prime}$) to generate enhanced features ($y_{0}$/$y$). $\odot$ denotes the Hadamard product and $\oplus$ denotes the addition by element.
• Figure 4: Rate-distortion performance curves aggregated over four test datasets. (a)/(b)/(c)/(d) and (e)/(f)/(g)/(h) are results on SID, SDSD-indoor, SDSD-outdoor, and SMID about PSNR and MS-SSIM, respectively. Remarkably, we are the first to address the problem of error accumulation and information loss in the joint task of image compression and low-light image enhancement, so there is no existing method for comparison. We adopt the low-light enhancement method xu2022snr for comparison. Experimental results obviously show that our proposed joint solution achieves great advantages compared to both "Compress before Enhance (CbE)" and "Enhance before Compress (EbC)" sequential solutions.
• Figure 5: Comparison of computational costs and model size. "TCM-S"/"TCM-M"/"TCM-L" represents the sequential solution of the 64/96/128 channels compression method liu2023learned before the low-light image enhancement method xu2022snr. "Cheng-S"/"Cheng-L" represents the sequential solution of the 128/192 channels compression method cheng2020learned before the low-light image enhancement method xu2022snr. Obviously, our joint solution has the advantage of lower computational costs and fewer model parameters.