Unsupervised Learning Based Multi-Scale Exposure Fusion

TL;DR

Novel loss functions are proposed for the ULMEF and they are defined by using all the images to be fused and other differently exposed images from the same HDR scene to guide the proposed ULMEF to learn more reliable information from the HDR scene.

Abstract

Unsupervised learning based multi-scale exposure fusion (ULMEF) is efficient for fusing differently exposed low dynamic range (LDR) images into a higher quality LDR image for a high dynamic range (HDR) scene. Unlike supervised learning, loss functions play a crucial role in the ULMEF. In this paper, novel loss functions are proposed for the ULMEF and they are defined by using all the images to be fused and other differently exposed images from the same HDR scene. The proposed loss functions can guide the proposed ULMEF to learn more reliable information from the HDR scene than existing loss functions which are defined by only using the set of images to be fused. As such, the quality of the fused image is significantly improved. The proposed ULMEF also adopts a multi-scale strategy that includes a multi-scale attention module to effectively preserve the scene depth and local contrast in the fused image. Meanwhile, the proposed ULMEF can be adopted to achieve exposure interpolation and exposure extrapolation. Extensive experiments show that the proposed ULMEF algorithm outperforms state-of-the-art exposure fusion algorithms.

Figures (8)

• Figure 1: Structure of the proposed MSF-NET. The MSF-NET is on top of a hierarchical structure with three level which is helpful to preserve scene depth and local contrast in a fused image and also improves the MEF-SSIM of the fused image.
• Figure 2: Multi Scale Recursive Residual Group (MSRRG), each MSRRG contains multi scale dual attention blocks (DAB). Each DAB contains spatial and channel attention modules.
• Figure 3: Visual comparison of ten different exposure fusion algorithms with the inputs as two LER images. The input data in the first column are from 1zheng2023. There are brightness order reversal artifacts in the fused images by the MEF 1mertens2007, GGIF kou2017, FMMEF li2020fast, PESPD zhang2023multi, MEFNet MEFNet, and MEFLUT jiang2023meflut.
• Figure 4: Fusion results for comparison of different fusion algorithms with images. The input data in the first columns are from Cai2018, the number of fused images is 2. It achieves stable fusion results across different domain datasets. There are halos in the fused images by the GGIF kou2017, MEFNet MEFNet, and MEFLUT jiang2023meflut. Information in the brightest regions is not preserved well by the MEF 1mertens2007, PESPD zhang2023multi, DeepFuse 1prabhakar2017, MEFGAN mefgan and FFMEF zheng2023efficient.
• Figure 5: Comparison among the proposed algorithm and the algorithms in MEF 1mertens2007, GGIF kou2017, FMMEF li2020fast, MEFNet MEFNet, MEFLUT jiang2023meflut when the inputs are three NER images. As illustrated by the highlighted parts, information in the brightest and darkest regions of HDR scenes is much more visible regardless of display by the proposed algorithm.