Event-assisted 12-stop HDR Imaging of Dynamic Scene

TL;DR

This work tackles pushing HDR imaging in dynamic scenes to 12 stops by combining an event camera with an RGB camera. It introduces an explicit event-assisted alignment module to robustly align LDR frames with large exposure gaps, and a diffusion-based fusion module that leverages priors from pretrained diffusion models to suppress alignment artifacts and preserve detail. The authors also present the ESHDR dataset, a paired RGB-Event dataset for 12-stop HDR, and validate the approach on simulated and real-world data, achieving state-of-the-art results on both fidelity and perceptual metrics. The approach has practical implications for high-contrast imaging in dynamic scenes and highlights the value of integrating event signals with diffusion priors to extend HDR capabilities.

Abstract

High dynamic range (HDR) imaging is a crucial task in computational photography, which captures details across diverse lighting conditions. Traditional HDR fusion methods face limitations in dynamic scenes with extreme exposure differences, as aligning low dynamic range (LDR) frames becomes challenging due to motion and brightness variation. In this work, we propose a novel 12-stop HDR imaging approach for dynamic scenes, leveraging a dual-camera system with an event camera and an RGB camera. The event camera provides temporally dense, high dynamic range signals that improve alignment between LDR frames with large exposure differences, reducing ghosting artifacts caused by motion. Also, a real-world finetuning strategy is proposed to increase the generalization of alignment module on real-world events. Additionally, we introduce a diffusion-based fusion module that incorporates image priors from pre-trained diffusion models to address artifacts in high-contrast regions and minimize errors from the alignment process. To support this work, we developed the ESHDR dataset, the first dataset for 12-stop HDR imaging with synchronized event signals, and validated our approach on both simulated and real-world data. Extensive experiments demonstrate that our method achieves state-of-the-art performance, successfully extending HDR imaging to 12 stops in dynamic scenes.

Figures (12)

• Figure 1: Visual comparisons of different HDR imaging methods on real-captured data. We utilize Vibrant tone mapping style in commercial HDR software (Photomatix) to better visualization.
• Figure 2: Illustration of the proposed framework for 12-stop HDR Imaging in Dynamic Scenes. The framework consists of two main components: (a) an event-assisted alignment module and (b) a diffusion-based fusion module.
• Figure 3: Ablation study on fusion and alignment modules by comparison of different configurations.
• Figure 4: Simulation pipeline of 12-stop HDR imaging with event.
• Figure 5: Visual comparisons of different HDR imaging methods. Since visualizing both the darkest and brightest areas in a 12-stop HDR image is a challenging task, we utilize two tone mapping styles, namely Deep and Vibrant, using commercial HDR software (Photomatix) to better highlight details in the bright and dark regions for comparison.