Exposure Completing for Temporally Consistent Neural High Dynamic Range Video Rendering

TL;DR

This work addresses HDR video rendering from LDR sequences with alternating exposures, where missing exposure information at each frame causes flicker and artifacts. It introduces NECHDR, a two-stage neural framework that first completes absent exposures by flow-guided interpolation of neighboring LDR frames and then performs coarse-to-fine HDR rendering guided by the completed features, with a blending network to fuse multiple HDR-domain inputs. The approach leverages pyramid features, flow-guided warping, and joint optimization to reduce ghosting and noise, achieving state-of-the-art results on standard benchmarks and demonstrating strong temporal consistency. The proposed exposure completing paradigm highlights a promising direction for HDR video rendering by ensuring complete exposure information at every time stamp, potentially broadening accessible HDR video synthesis from LDR content.

Abstract

High dynamic range (HDR) video rendering from low dynamic range (LDR) videos where frames are of alternate exposure encounters significant challenges, due to the exposure change and absence at each time stamp. The exposure change and absence make existing methods generate flickering HDR results. In this paper, we propose a novel paradigm to render HDR frames via completing the absent exposure information, hence the exposure information is complete and consistent. Our approach involves interpolating neighbor LDR frames in the time dimension to reconstruct LDR frames for the absent exposures. Combining the interpolated and given LDR frames, the complete set of exposure information is available at each time stamp. This benefits the fusing process for HDR results, reducing noise and ghosting artifacts therefore improving temporal consistency. Extensive experimental evaluations on standard benchmarks demonstrate that our method achieves state-of-the-art performance, highlighting the importance of absent exposure completing in HDR video rendering. The code is available at https://github.com/cuijiahao666/NECHDR.

Figures (13)

• Figure 1: Pipline of the proposed NECHDR network. Our network mainly consists of three processes: feature encoding for LDRs, exposure completing for LDR frames with missing exposures, and HDR rendering. We extract pyramid features from the input LDR frames using a parameter shared feature encoder. Then, the optical flows predicted by the lower-level HDR rendering decoder is used to warp neighbor frames to the reference frame. Subsequently, the exposure completing decoder performs exposure completing based on the warped neighbor frame features. The input features and the completed feature are fed into the HDR rendering decoder to render the coarse HDR frame. Finally, the original frames, the completed frame, and the coarse HDR frame are fused together through a simple blending network to produce a high-quality HDR result.
• Figure 2: Qualitative comparison in scenes with over-saturation and motion. Left: 2-Exposure scene from the Cinematic Video dataset froehlich2014creating. Right: 2-Exposure scene from the HDRVideo dataset kalantari2013patch."EC" refers to our exposure completion results.
• Figure 3: Qualitative comparison in scenes with noise and motion. Left: 2-Exposure scene from the Cinematic Video dataset froehlich2014creating. Right: 2-Exposure scene from the DeepHDRVideo dataset chen2021hdr. "EC" refers to our exposure completion results.
• Figure 4: Qualitative comparison of the models corresponding to the ablation study on a dynamic scene of DeepHDRVideo chen2021hdr dataset.
• Figure 5: Visual comparisons of temporal consistency.