Single-Image HDR Reconstruction Assisted Ghost Suppression and Detail Preservation Network for Multi-Exposure HDR Imaging

TL;DR

This work tackles HDR reconstruction in dynamic scenes by jointly exploiting single-frame and multi-exposure cues. It introduces a dual-branch network with SHDR-ESI for ghost-free single-frame HDR and SHDR-A-MHDR for ghost-suppressed multi-exposure fusion, augmented by DEMs (SRM,MRM), FIFM, and GSM to preserve and transfer detail. The approach yields state-of-the-art results across four public datasets, with notable improvements in ghost suppression and saturated-region detail recovery, validated by extensive ablations and parameter analyses. The method offers practical impact by enabling high-quality HDR reconstruction from standard LDR sequences in motion-rich scenarios, with potential for real-world photography and video applications.

Abstract

The reconstruction of high dynamic range (HDR) images from multi-exposure low dynamic range (LDR) images in dynamic scenes presents significant challenges, especially in preserving and restoring information in oversaturated regions and avoiding ghosting artifacts. While current methods often struggle to address these challenges, our work aims to bridge this gap by developing a multi-exposure HDR image reconstruction network for dynamic scenes, complemented by single-frame HDR image reconstruction. This network, comprising single-frame HDR reconstruction with enhanced stop image (SHDR-ESI) and SHDR-ESI-assisted multi-exposure HDR reconstruction (SHDRA-MHDR), effectively leverages the ghost-free characteristic of single-frame HDR reconstruction and the detail-enhancing capability of ESI in oversaturated areas. Specifically, SHDR-ESI innovatively integrates single-frame HDR reconstruction with the utilization of ESI. This integration not only optimizes the single image HDR reconstruction process but also effectively guides the synthesis of multi-exposure HDR images in SHDR-AMHDR. In this method, the single-frame HDR reconstruction is specifically applied to reduce potential ghosting effects in multiexposure HDR synthesis, while the use of ESI images assists in enhancing the detail information in the HDR synthesis process. Technically, SHDR-ESI incorporates a detail enhancement mechanism, which includes a self-representation module and a mutual-representation module, designed to aggregate crucial information from both reference image and ESI. To fully leverage the complementary information from non-reference images, a feature interaction fusion module is integrated within SHDRA-MHDR. Additionally, a ghost suppression module, guided by the ghost-free results of SHDR-ESI, is employed to suppress the ghosting artifacts.

Figures (15)

• Figure 1: (a) Reference image. (b) Enhancement stop image (ESI). ESI has the capability to highlight subtle information in oversaturated regions of the reference image.
• Figure 2: Overall framework of the proposed method. (a) the main network consists of the SHDR-ESI branch and the SHDR-A-MHDR branch, where SHDR-ESI primarily assists SHDR-A-MHDR in suppressing ghosting artifacts within the fused features. SRM and MRM are employed to highlight and aggregate valuable information from both the reference image and ESI, within the SHDR-ESI branch. In the SHDR-A-MHDR branch, FIFM is designed to merge the information from input images and enhance the role of valuable information in HDR image reconstruction. Subsequently, GSM is employed to mitigate potential ghosting artifacts within the fused features. (b) the feature extraction network $E_{i} (i=1,2,3,4)$ is composed of a $3\times 3$ convolution and two CA-ViT Liu-2022-Ghost-free. (c) R-Subblock consists of three residual blocks (Res-blocks).
• Figure 3: Illustration of the self-representation module (SRM).
• Figure 4: Illustration of the mutual-representation module (MRM).
• Figure 5: Detailed structure of the feature interaction fusion module (FIFM).