Efficient HDR Reconstruction from Real-World Raw Images

TL;DR

The paper tackles efficient HDR reconstruction on resource-limited devices by operating in the raw domain and avoiding heavy alignment modules. It introduces RepUNet, a lightweight dual-encoder network augmented with a Topological Convolution Block and re-parameterization to enable real-time 4K HDR on mobile hardware. A novel data formation pipeline yields RealRaw-HDR, a large raw SDR-HDR dataset derived from dual-exposure sensor physics, enabling robust training and evaluation. The proposed alignment-free, motion-aware loss further suppresses ghosting in dynamic scenes. Empirically, RepUNet matches or exceeds state-of-the-art HDR quality while using substantially less computation and memory, offering practical deployment advantages for edge devices.

Abstract

The growing prevalence of high-resolution displays on edge devices has created a pressing need for efficient high dynamic range (HDR) imaging algorithms. However, most existing HDR methods either struggle to deliver satisfactory visual quality or incur high computational and memory costs, limiting their applicability to high-resolution inputs (typically exceeding 12 megapixels). Furthermore, current HDR dataset collection approaches are often labor-intensive and inefficient. In this work, we explore a novel and practical solution for HDR reconstruction directly from raw sensor data, aiming to enhance both performance and deployability on mobile platforms. Our key insights are threefold: (1) we propose RepUNet, a lightweight and efficient HDR network leveraging structural re-parameterization for fast and robust inference; (2) we design a new computational raw HDR data formation pipeline and construct a new raw HDR dataset, RealRaw-HDR; (3) we design a plug-and-play motion alignment loss to suppress ghosting artifacts under constrained bandwidth conditions effectively. Our model contains fewer than 830K parameters and takes less than 3 ms to process an image of 4K resolution using one RTX 3090 GPU. While being highly efficient, our model also achieves comparable performance to state-of-the-art HDR methods in terms of PSNR, SSIM, and a color difference metric.

Figures (12)

• Figure 1: The raw SDR-HDR pair formation pipeline. Two clean HDR raw images, $I_1$ and $I_2$, have been processed through black-level correction and normalization. After manual digital gain, clip, add noise, and normalization, the long-exposure image $I_l$ is overexposed in the bright areas, and the short-exposure image $I_s$ has dark area information covered by noise.
• Figure 2: Real samples collected by the proposed raw SDR-HDR pair formation pipeline. For display purposes, we do not apply luminance alignment processing.
• Figure 3: Illustration of (a) Base Model and (b) Topology Convolution Block (TCB). In the training phase, the TCB employs multiple branches, which can be merged into one normal convolution layer in the inference stage.
• Figure 4: An illustrative sample of data construction for the proposed alignment-free and motion-aware short-exposure-first selection loss.
• Figure 5: Visual comparison of state-of-the-art HDR reconstruction methods on our synthetic dataset from RealRaw-HDR.