Deep Lightweight Unrolled Network for High Dynamic Range Modulo Imaging

TL;DR

This work tackles HDR reconstruction from modulo-imaged measurements by formulating an optimization problem that merges a spatial finite-difference fidelity term with a deep learnable prior, then unrolling the ADMM iterations into a lightweight neural network. The model uses a compact denoiser within each layer to impose learned priors and achieves fast inference with strong noise robustness; it also introduces Scaling Equivariance for self-supervised fine-tuning to adapt to out-of-distribution modulo data. Empirical results on the UnModNet HDR dataset show up to several dB gains in PSNR and improved perceptual quality (HDR-VDP-3) across noise levels, while real-modulo experiments demonstrate enhanced artifact suppression and detail preservation. The combination of unrolled optimization, a lightweight architecture, and SE-based adaptation offers a practical, scalable solution for HDR reconstruction in modulo-imaging systems with real-world applicability.

Abstract

Modulo-Imaging (MI) offers a promising alternative for expanding the dynamic range of images by resetting the signal intensity when it reaches the saturation level. Subsequently, high-dynamic range (HDR) modulo imaging requires a recovery process to obtain the HDR image. MI is a non-convex and ill-posed problem where recent recovery networks suffer in high-noise scenarios. In this work, we formulate the HDR reconstruction task as an optimization problem that incorporates a deep prior and subsequently unrolls it into an optimization-inspired deep neural network. The network employs a lightweight convolutional denoiser for fast inference with minimal computational overhead, effectively recovering intensity values while mitigating noise. Moreover, we introduce the Scaling Equivariance term that facilitates self-supervised fine-tuning, thereby enabling the model to adapt to new modulo images that fall outside the original training distribution. Extensive evaluations demonstrate the superiority of our method compared to state-of-the-art recovery algorithms in terms of performance and quality.

Figures (10)

• Figure 1: Modulo imaging. (a) Images from conventional saturated image acquisition as CCD. (b) Images from modulo sensor that resets the pixel intensity. (c) Recovered image from the modulo image. (d) Horizontal intensity cross-sections of each blue component image (red dots line) are provided to evidence the sensing and recovery process.
• Figure 2: (a) The signal $\boldsymbol{x}$ is decomposed as $\boldsymbol{x = y} + \boldsymbol{k} \cdot 2^b$. (b) (b) The application of spatial different operator ($\Delta$) to each term. Notice that each term of $\Delta \boldsymbol{k}$ is 0 except for the values where the reset occurs, which have values of $\pm 2^{b}$. (c) The results of applying the centered modulo operator $\mathcal{M}_b$ to each term. Notice that $\Delta \boldsymbol{x} = \mathcal{M}_b(\Delta \boldsymbol{y})$.
• Figure 3: Deep Unrolling Architecture: The centered modulo operator is applied to the image, which is then passed through $T$ learned block layers inspired by the ADMM formulation. The HDR image is reconstructed as the output of the final layer.
• Figure 4: Visual Results of HDR Recovery Methods. Qualitative comparison of unwrapped images from various methods, demonstrating the proposed Unrolled approach achieves superior visual fidelity and quantitative metrics (PSNR$\vert$SSIM) compared to state-of-the-art alternatives. Visual HDR images are displayed using Reinhard tone mapping on luminance component reinhard2023photographic.
• Figure 5: Out of distribution evaluation. Visual comparison of Unrolled and UnModNet on real RGB captures