FlexHDR: Modelling Alignment and Exposure Uncertainties for Flexible HDR Imaging

TL;DR

The paper tackles ghosting and exposure errors in multi-frame HDR by introducing an HDR-aware, uncertainty-driven fusion method that jointly handles alignment and exposure reliability. It presents an HDR-specific optical flow network that shares information across frames, learnable exposure and alignment uncertainty modules, and a permutation-invariant, multi-stage fusion that can process any number of input LDR images. Across ablations and benchmarks, the approach achieves consistent improvements in PSNR, SSIM, and perceptual metrics, including up to 1.1 dB PSNR-PU over prior methods and superior HDR-VDP-2 scores, while maintaining robust performance across variable input counts. The work advances practical HDR imaging by enabling flexible input sets, reducing ghosting, and delivering higher-quality detail and color under challenging lighting and motion conditions.

Abstract

High dynamic range (HDR) imaging is of fundamental importance in modern digital photography pipelines and used to produce a high-quality photograph with well exposed regions despite varying illumination across the image. This is typically achieved by merging multiple low dynamic range (LDR) images taken at different exposures. However, over-exposed regions and misalignment errors due to poorly compensated motion result in artefacts such as ghosting. In this paper, we present a new HDR imaging technique that specifically models alignment and exposure uncertainties to produce high quality HDR results. We introduce a strategy that learns to jointly align and assess the alignment and exposure reliability using an HDR-aware, uncertainty-driven attention map that robustly merges the frames into a single high quality HDR image. Further, we introduce a progressive, multi-stage image fusion approach that can flexibly merge any number of LDR images in a permutation-invariant manner. Experimental results show our method can produce better quality HDR images with up to 1.1dB PSNR improvement to the state-of-the-art, and subjective improvements in terms of better detail, colours, and fewer artefacts.

Figures (17)

• Figure 1: Our method, intermediate results, and comparisons. LDR input images are shown on the left and our tone mapped result is in the centre. The visualisations on the right are: I Input Image, II Exposure Map, III Attention Map. The bottom row compares to several state-of-the-art methods. Our uncertainty modelling more effectively handles regions of overexposure and motion between input frames.
• Figure 2: Model Architecture. Our model accepts any number of LDR images as input and aligns them with a HDR flow network which shares information between frames with pooling operations. We then model exposure and alignment uncertainties which are used by our attention network to suppress untrustworthy regions. Finally, the merging network consists of a grouped residual dense block with multi-stage max-pooling operations for gradual merging of input frames.
• Figure 3: Architecture of our HDR Iterative Optical Flow Network. The feature encoder first downsamples the input features by 8x before the recurrent convolutions iteratively refine the estimated optical flow field.
• Figure 4: We model exposure uncertainty as a piecewise linear function where $\alpha$ and $\beta$ are predicted by the network. Given the mean pixel values of an image, $\hat{I}_i$, our model predicts an image specific response of exposure confidence, $E_i$.
• Figure 5: An overview of our multi-stage fusion mechanism. The green arrows show the direction of information flow between the different streams. By sharing information between streams at multiple points, the network is able to produce clear, detailed images.