MobileMEF: Fast and Efficient Method for Multi-Exposure Fusion

TL;DR

This work proposes a new method for multi-exposure fusion based on an encoder-decoder deep learning architecture with efficient building blocks tailored for mobile devices that outperforms state-of-the-art techniques regarding full-reference quality measures and computational efficiency, making it ideal for real-time applications on hardware-constrained devices.

Abstract

Recent advances in camera design and imaging technology have enabled the capture of high-quality images using smartphones. However, due to the limited dynamic range of digital cameras, the quality of photographs captured in environments with highly imbalanced lighting often results in poor-quality images. To address this issue, most devices capture multi-exposure frames and then use some multi-exposure fusion method to merge those frames into a final fused image. Nevertheless, most traditional and current deep learning approaches are unsuitable for real-time applications on mobile devices due to their heavy computational and memory requirements. We propose a new method for multi-exposure fusion based on an encoder-decoder deep learning architecture with efficient building blocks tailored for mobile devices. This efficient design makes our model capable of processing 4K resolution images in less than 2 seconds on mid-range smartphones. Our method outperforms state-of-the-art techniques regarding full-reference quality measures and computational efficiency (runtime and memory usage), making it ideal for real-time applications on hardware-constrained devices. Our code is available at: https://github.com/LucasKirsten/MobileMEF.

Figures (7)

• Figure 1: Comparison of computational cost and performance of SOTA MEF methods: HoLoCo holoco, IFCNN ifcnn, MEFLUT meflut, SAMT-MEF samt_mef, TransMEF transmef. MobileMEF can process 4k images in 1.82 seconds on a mid-range mobile device using GPU.
• Figure 2: Overall proposed MobileMEF model architecture. The encoder-decoder model receives an input with $C$ channels and $K$ frames and returns the fused frames. The SSF module merges the $K$ input frames and adds it to the model's output.
• Figure 3: IRA Block as proposed in LPIENet lpienet.
• Figure 4: Visual comparison of MobileMEF with SOTA methods using EV +1 and -1 as input frames.
• Figure 5: Visual comparison of MobileMEF with SOTA methods using the most under and over exposed input frames.