Little Strokes Fell Great Oaks: Boosting the Hierarchical Features for Multi-exposure Image Fusion

TL;DR

The paper addresses the challenge of limited dynamic range in multi-exposure image fusion by proposing BHF-MEF, an unsupervised architecture that boosts hierarchical features through a Gamma Correction Module (GCM), a Shadow Encoder/Decoder with transformer blocks, a Texture Enhancement Module (TEM), and a Color Enhancement (CE) step. It introduces a two-part loss combining gamma-correction and fusion objectives and validates the approach with extensive experiments across multiple MEF datasets, showing superior quantitative and qualitative performance. The method leverages latent information in source images, preserves textures, and enhances color saturation while retaining details, with source code available for reproducibility. This work advances unsupervised MEF by integrating latent-detail exploitation, global-local feature fusion, and colorfulness improvements in a cohesive framework.

Abstract

In recent years, deep learning networks have made remarkable strides in the domain of multi-exposure image fusion. Nonetheless, prevailing approaches often involve directly feeding over-exposed and under-exposed images into the network, which leads to the under-utilization of inherent information present in the source images. Additionally, unsupervised techniques predominantly employ rudimentary weighted summation for color channel processing, culminating in an overall desaturated final image tone. To partially mitigate these issues, this study proposes a gamma correction module specifically designed to fully leverage latent information embedded within source images. Furthermore, a modified transformer block, embracing with self-attention mechanisms, is introduced to optimize the fusion process. Ultimately, a novel color enhancement algorithm is presented to augment image saturation while preserving intricate details. The source code is available at https://github.com/ZhiyingDu/BHFMEF.

Figures (8)

• Figure 1: Workflow comparison of our proposed method with existing multi-exposure fusion approaches.
• Figure 2: The architecture of the proposed method. The network consists of a gamma correction module (GCM), a denoise module (DN), a texture enhance module (TEM), a shadow encoder, a decoder. TB: transformer block.
• Figure 3: Visual comparison of our method with other state-of-the-art methods on datasets cai2018learningzhang2021benchmarkinggallo2009artifactma2015perceptual. Our method performs the best visual quality in terms of natural color and details.
• Figure 4: Visual display of GCM output results in important areas of source images. It includes three types of images: the GCM iteration result, the gradient image, and the image representing the difference between the original pixel values and 128.
• Figure 5: Visual comparison of ablation experiments on TEM. The second row is the corresponding gradient-based details of the first row. The number I,II,III,IV,V here corresponds to I,II,III,IV,V in Figure \ref{['fig:Ablation_TEM']}.