Dream-IF: Dynamic Relative EnhAnceMent for Image Fusion

TL;DR

This work addresses robust multi-modal image fusion under diverse degradations by modeling the intrinsic correlation between fusion and enhancement through a Dynamic Relative Enhancement (Dream-IF) framework. Dream-IF computes a Relative Dominance signal to guide cross-modal and self-enhancement, enabling restoration-aware fusion without explicit degradation priors, and employs prompt-based degradation encoding to adapt restoration prompts. The approach achieves state-of-the-art performance on LLVIP, MSRS, and M3FD, demonstrates robustness under Gaussian, Poisson, speckle, and rainy degradations, and improves downstream object detection performance. The findings highlight the practical value of leveraging dominance-driven enhancement to bolster fusion quality in real-world, degraded multi-modal imaging scenarios, with code publicly available.

Abstract

Image fusion aims to integrate comprehensive information from images acquired through multiple sources. However, images captured by diverse sensors often encounter various degradations that can negatively affect fusion quality. Traditional fusion methods generally treat image enhancement and fusion as separate processes, overlooking the inherent correlation between them; notably, the dominant regions in one modality of a fused image often indicate areas where the other modality might benefit from enhancement. Inspired by this observation, we introduce the concept of dominant regions for image enhancement and present a Dynamic Relative EnhAnceMent framework for Image Fusion (Dream-IF). This framework quantifies the relative dominance of each modality across different layers and leverages this information to facilitate reciprocal cross-modal enhancement. By integrating the relative dominance derived from image fusion, our approach supports not only image restoration but also a broader range of image enhancement applications. Furthermore, we employ prompt-based encoding to capture degradation-specific details, which dynamically steer the restoration process and promote coordinated enhancement in both multi-modal image fusion and image enhancement scenarios. Extensive experimental results demonstrate that Dream-IF consistently outperforms its counterparts. The code is publicly available.\footnote{ https://github.com/jehovahxu/Dream-IF

Figures (5)

• Figure 1: Image fusion (IF) and image restoration (IR) task paradigms. (a) Joint cascaded for restoration followed by fusion, (b) Directly integrate fusion and restoration, (c) Mutually enhanced fusion and restoration through the inherent correlations.
• Figure 2: An overview of the proposed Dream-IF. We introduce the Relative Enhancement block, which implicitly enhances non-dominant representations by leveraging relative information during the fusion and restoration process. This module captures the relative dominance inherent in the complementary nature of the image fusion model and uses it to facilitate both cross and self enhancement, ultimately producing the enhanced feature.
• Figure 3: Qualitative comparisons of various methods on representative images selected from the LLVIP, MSRS and M3FD datasets.
• Figure 4: Qualitative comparisons of various methods under degradation on the LLVIP dataset. SCUNet+ refers to the application of the SCUNet model for image restoration prior to the fusion process.
• Figure 5: Visualization of comparison in one-stage restoration.