AutoLUT: LUT-Based Image Super-Resolution with Automatic Sampling and Adaptive Residual Learning

TL;DR

This work tackles efficient image super-resolution on edge devices by enhancing LUT-based SR with two plug-and-play modules: Automatic Sampling (AutoSample), which learns pixel sampling to yield adaptive receptive fields without increasing LUT size, and Adaptive Residual Learning (AdaRL), which uses spatially varying residual fusion to improve inter-layer information flow. The AutoLUT framework replaces fixed LUT groups with AutoLUT groups that combine AutoSample and AdaRL before the final LUT, enabling flexible sampling and robust residual connections. Across MuLUT and SPF-LUT baselines, the approach yields PSNR improvements (e.g., about +0.20 dB on MuLUT) while substantially reducing storage and inference time (over 50% storage reduction and ~2/3 faster inference on SPF-LUT; edge-device tests show large speedups with maintained or improved quality). The method is plug-and-play, scales with sampling size and branch count, and demonstrates strong practical impact for deployable SR on resource-limited hardware; code is publicly available.

Abstract

In recent years, the increasing popularity of Hi-DPI screens has driven a rising demand for high-resolution images. However, the limited computational power of edge devices poses a challenge in deploying complex super-resolution neural networks, highlighting the need for efficient methods. While prior works have made significant progress, they have not fully exploited pixel-level information. Moreover, their reliance on fixed sampling patterns limits both accuracy and the ability to capture fine details in low-resolution images. To address these challenges, we introduce two plug-and-play modules designed to capture and leverage pixel information effectively in Look-Up Table (LUT) based super-resolution networks. Our method introduces Automatic Sampling (AutoSample), a flexible LUT sampling approach where sampling weights are automatically learned during training to adapt to pixel variations and expand the receptive field without added inference cost. We also incorporate Adaptive Residual Learning (AdaRL) to enhance inter-layer connections, enabling detailed information flow and improving the network's ability to reconstruct fine details. Our method achieves significant performance improvements on both MuLUT and SPF-LUT while maintaining similar storage sizes. Specifically, for MuLUT, we achieve a PSNR improvement of approximately +0.20 dB improvement on average across five datasets. For SPF-LUT, with more than a 50% reduction in storage space and about a 2/3 reduction in inference time, our method still maintains performance comparable to the original. The code is available at https://github.com/SuperKenVery/AutoLUT.

Figures (6)

• Figure 1: Performance-storage trade-offs for ×4 super-resolution on Set5 DBLP:conf/bmvc/BevilacquaRGA12 compared with MuLUT Li_2022_MuLUT and SPF-LUT+DFC Li_2024_CVPR. As shown, our method can achieve the same level of performance at much lower storage requirement or much better performance with the same storage.
• Figure 2: Overview of the AutoLUT-based SR framework. The figure illustrates the process of enhancing LUT-based super-resolution models by integrating the AutoLUT. (a) The AutoLUT-based SR framework replaces traditional LUT Group with AutoLUT Group for improved flexibility. (b) The AutoLUT Group can flexibly adjust the number of branches according to the requirements, enabling more efficient processing of diverse information. (c) The AutoLUT, where the inputs $X_{n-1}$ and $X_{n-2}$ pass through the AutoSample, followed by a combination using learnable residual weights. The final output is processed through the basic LUT.
• Figure 3: Comparison of sampling strategies on $\times$2 super-resolution. (a) A LUT is a data structure composed of index-value pairs, the sampled indices are used to retrieve their corresponding values. (b) The sampling methods in MuLUT Li_2022_MuLUT and SPF-LUT Li_2024_CVPR, use three distinct sampling strategies, each selecting four fixed pixels from a $3 \times 3$ input window. Together, these three sampling configurations fully cover the $3 \times 3$ window. After applying a rotation ensemble, where the results from four rotated versions are averaged, the receptive field expands from $3 \times 3$ to $5 \times 5$. The pixel $I_0$ in the low-resolution image corresponds to pixel $[ V_{0}^0, V_{0}^1, V_{0}^2, V_{0}^3 ]$ in the image after super-resolution. (c) The learnable AutoSample strategy selects pixels by learning sampling weights through convolution during training, using pixel shuffle to expand the receptive field to $(2k-1) \times (2k-1)$. Finally, the desired index is obtained by applying AdaRL.
• Figure 4: Receptive field and storage comparisons of LUTs. The size of the bubble represents the storage size.
• Figure 5: Qualitative comparison for $\times$4 super-resolution on benchmark datasets. Our methods are in bold.

Theorems & Definitions (1)

• Theorem 1