DnLUT: Ultra-Efficient Color Image Denoising via Channel-Aware Lookup Tables

TL;DR

This work introduces DnLUT, a LUT-based framework for ultra-efficient color image denoising that targets edge devices. It combines a Pairwise Channel Mixer to capture channel-spatial correlations with an L-shaped convolution to enlarge the receptive field while enabling 3D LUTs, dramatically reducing storage. After training a DnNet, all possible inputs are cached into LUTs, enabling fast, low-energy inference that outperforms existing LUT methods by over 1 dB CPSNR and remains orders of magnitude more efficient than CNNs. The PCM module also serves as a plug-in to boost other LUT methods, and extensive experiments on Gaussian and real-world datasets demonstrate substantial gains in both quality and efficiency, making edge deployment practical for color image denoising.

Abstract

While deep neural networks have revolutionized image denoising capabilities, their deployment on edge devices remains challenging due to substantial computational and memory requirements. To this end, we present DnLUT, an ultra-efficient lookup table-based framework that achieves high-quality color image denoising with minimal resource consumption. Our key innovation lies in two complementary components: a Pairwise Channel Mixer (PCM) that effectively captures inter-channel correlations and spatial dependencies in parallel, and a novel L-shaped convolution design that maximizes receptive field coverage while minimizing storage overhead. By converting these components into optimized lookup tables post-training, DnLUT achieves remarkable efficiency - requiring only 500KB storage and 0.1% energy consumption compared to its CNN contestant DnCNN, while delivering 20X faster inference. Extensive experiments demonstrate that DnLUT outperforms all existing LUT-based methods by over 1dB in PSNR, establishing a new state-of-the-art in resource-efficient color image denoising. The project is available at https://github.com/Stephen0808/DnLUT.

Figures (7)

• Figure 1: Model comparison in terms of color peak signal-to-noise ratio (CPSNR), runtime, and storage. The CPSNR and runtime are calculated on the CBSD68 dataset with Gaussian noise level $\sigma = 25$ using Qualcomm Snapdragon 8 Gen2. Our method outperforms state-of-the-art LUT-based methods with the highest CPSNR, at a low storage requirements and reduced runtime. Additionally, our PCM module serves as a versatile plug-in module that enhances existing methods' performance by over 1dB.
• Figure 2: System architecture of DnLUT: (a) The DnNet pipeline integrates pairwise channel mixers and L-shaped convolutions, with multi-scale fusion enhancing receptive field coverage. Channel dimensions are flattened for parallel processing in L-shaped operations, then unfolded for PCM input. (b) Input pixels undergo four rotations (0°, 90°, 180°, 270°) during processing, with outputs averaged for enhanced results. (c) Post-training, all possible input combinations are processed through DnNet modules, with outputs cached in optimized 3D or 4D LUTs. (d) During inference, input pixels are efficiently processed through multiple LUTs, with each LUT's outputs informing subsequent LUT indices, culminating in final denoised pixel values.
• Figure 3: Taxonomy of kernel patterns for LUT-based methods. Dark cubes indicate rotation points, while medium-dark regions show involved pixel positions during one rotation.
• Figure 4: Comparison of spatial-wise kernel designs. Left patterns show kernel configurations, while right tables quantify lookup frequencies during output retrieval.
• Figure 5: Qualitative comparison on synthetic datasets.