Capturing Stable HDR Videos Using a Dual-Camera System

TL;DR

An asynchronous dual-camera system (DCS) is designed, which enables independent exposure control across two cameras, eliminating the need for synchronization typically required in traditional multi-camera setups, and an exposure-adaptive fusion network (EAFNet) is formulated for the DCS system.

Abstract

High Dynamic Range (HDR) video acquisition using the alternating exposure (AE) paradigm has garnered significant attention due to its cost-effectiveness with a single consumer camera. However, despite progress driven by deep neural networks, these methods remain prone to temporal flicker in real-world applications due to inter-frame exposure inconsistencies. To address this challenge while maintaining the cost-effectiveness of the AE paradigm, we propose a novel learning-based HDR video generation solution. Specifically, we propose a dual-stream HDR video generation paradigm that decouples temporal luminance anchoring from exposure-variant detail reconstruction, overcoming the inherent limitations of the AE paradigm. To support this, we design an asynchronous dual-camera system (DCS), which enables independent exposure control across two cameras, eliminating the need for synchronization typically required in traditional multi-camera setups. Furthermore, an exposure-adaptive fusion network (EAFNet) is formulated for the DCS system. EAFNet integrates a pre-alignment subnetwork that aligns features across varying exposures, ensuring robust feature extraction for subsequent fusion, an asymmetric cross-feature fusion subnetwork that emphasizes reference-based attention to effectively merge these features across exposures, and a reconstruction subnetwork to mitigate ghosting artifacts and preserve fine details. Extensive experimental evaluations demonstrate that the proposed method achieves state-of-the-art performance across various datasets, showing the remarkable potential of our solution in HDR video reconstruction. The codes and data captured by DCS will be available at https://zqqqyu.github.io/DCS-HDR/.

Figures (14)

• Figure 1: Overview of the alternating exposure (AE) paradigm and our proposed dual-stream paradigm. (a) existing AE paradigm fuses alternating exposures (LE/ME/HE) with varying reference frames, resulting in flickering artifacts (luminance fluctuations exceeding 30). (b-d) Our dual-stream paradigm employs a fixed-exposure reference sequence for stable luminance anchoring, while a separate stream handles exposure-variant enhancement, ensuring consistent illumination across reference frames, enabling temporally consistent HDR reconstruction results (luminance fluctuations limited to under 1).
• Figure 2: Visualization of our dual-camera system. The primary camera captures continuous medium-exposure sequences as reference for temporal consistency, while the secondary camera alternates between low- and high-exposure to provide complementary information for reconstruction.
• Figure 3: The architecture of EAFNet consists of a pre-alignment subnetwork, an asymmetric cross-feature fusion subnetwork, and a restoration subnetwork. We introduce GLA and EFSM to leverage exposure information, explore the intrinsic properties of the images, and help preserve finer details across varying exposures. The asymmetric cross-feature fusion subnetwork improves image fusion by aligning cross-scale features and performing cross-feature fusion. The restoration subnetwork adopts a multi-scale architecture to reduce ghosting and refine features at different resolutions.
• Figure 4: The pre-alignment subnetwork is composed of two parts: global luminance alignment (GLA) and exposure-guided feature selection module (EFSM). The input is divided into multiple scales using a $3\times3$ convolution, with a shared-weight feature selection block applied at each scale.
• Figure 5: The structure of the asymmetric cross-feature fusion block. It employs asymmetric cross-feature attention to align reference and non-reference features, integrating coarse-to-fine guidance for enhanced reference-dominated feature fusion and improved HDR reconstruction.