Generating Content for HDR Deghosting from Frequency View

TL;DR

This work tackles ghosting in HDR reconstruction from dynamic scenes by introducing LF-Diff, a latent-space diffusion framework that learns compact low-frequency priors (LPR) to guide HDR synthesis. The method combines a regression-based Dynamic HDR Reconstruction Network (DHRNet) with LPENet-based priors and a diffusion model trained in two stages: first to learn the LPR from ground-truth HDRs, then to estimate the LPR from LDR inputs, with joint optimization to produce high-quality HDR results efficiently. Experiments on Kalantari2017Deep and Hu2020sensor show LF-Diff achieves state-of-the-art performance with notable speedups over prior diffusion-based approaches, while ablations confirm the efficacy of the diffusion prior, joint training, and latent-space diffusion for reducing ghosting artifacts. The approach offers a practical HDR deghosting solution for dynamic scenes, combining diffusion-model strengths with a lightweight latent representation to enable faster, perceptually faithful HDR reconstructions.

Abstract

Recovering ghost-free High Dynamic Range (HDR) images from multiple Low Dynamic Range (LDR) images becomes challenging when the LDR images exhibit saturation and significant motion. Recent Diffusion Models (DMs) have been introduced in HDR imaging field, demonstrating promising performance, particularly in achieving visually perceptible results compared to previous DNN-based methods. However, DMs require extensive iterations with large models to estimate entire images, resulting in inefficiency that hinders their practical application. To address this challenge, we propose the Low-Frequency aware Diffusion (LF-Diff) model for ghost-free HDR imaging. The key idea of LF-Diff is implementing the DMs in a highly compacted latent space and integrating it into a regression-based model to enhance the details of reconstructed images. Specifically, as low-frequency information is closely related to human visual perception we propose to utilize DMs to create compact low-frequency priors for the reconstruction process. In addition, to take full advantage of the above low-frequency priors, the Dynamic HDR Reconstruction Network (DHRNet) is carried out in a regression-based manner to obtain final HDR images. Extensive experiments conducted on synthetic and real-world benchmark datasets demonstrate that our LF-Diff performs favorably against several state-of-the-art methods and is 10$\times$ faster than previous DM-based methods.

Figures (6)

• Figure 1: Average PSNR vs. inference time on the Kalantari's dataset Kalantari2017Deep. Our method performs favorably and is 10$\times$ faster than the previous diffusion-model-based method DiffHDR 10288540.
• Figure 2: The proposed LF-Diff comprises DHRNet, LPRNet, and a denoising network. LF-Diff undergoes two training stages: Pretrain LF-Diff (Sec. \ref{['Stage One']}) and DM training (Sec. \ref{['Stage Two']}). Notably, during the inference stage, we do not input the ground-truth image into LPENet$_{DM}$ and the denoising networks. Instead, we solely utilize the reverse process of DMs.
• Figure 3: The architecture of (a) Feature Refinement Module (FRM) and (b) Cross-attention mechanism.
• Figure 4: Visual comparisons are conducted on testing data from various datasets, focusing on zoomed-in local areas of the HDR images estimated by our method and the compared techniques. Our model demonstrates the ability to generate HDR images of superior quality.
• Figure 5: Visual comparisons on the datasets without ground truth.