Reversible Efficient Diffusion for Image Fusion

TL;DR

We address the challenge of detail preservation in diffusion-based multi-modal image fusion by introducing Reversible Efficient Diffusion (RED), an explicitly supervised, memory-efficient framework that enables end-to-end training. RED leverages reversible fusion and reversible residual blocks to bypass the traditional memory bottlenecks of diffusion models, while aligning the diffusion process with fusion objectives through direct supervision from source images. The method reformulates DDIM sampling as a layered network, uses a learnable fusion weight, and optimizes a composite loss that emphasizes structure, pixel integrity, and edges, achieving state-of-the-art results across visible–infrared and medical fusion datasets and boosting downstream tasks like object detection. This approach delivers high-fidelity, modality-consistent fused images with favorable memory and computational characteristics, offering practical impact for real-world fusion scenarios and guiding future exploration of end-to-end diffusion-based fusion with controllable computation.

Abstract

Multi-modal image fusion aims to consolidate complementary information from diverse source images into a unified representation. The fused image is expected to preserve fine details and maintain high visual fidelity. While diffusion models have demonstrated impressive generative capabilities in image generation, they often suffer from detail loss when applied to image fusion tasks. This issue arises from the accumulation of noise errors inherent in the Markov process, leading to inconsistency and degradation in the fused results. However, incorporating explicit supervision into end-to-end training of diffusion-based image fusion introduces challenges related to computational efficiency. To address these limitations, we propose the Reversible Efficient Diffusion (RED) model - an explicitly supervised training framework that inherits the powerful generative capability of diffusion models while avoiding the distribution estimation.

Figures (12)

• Figure 1: Illustration comparing the proposed RED framework with previous fusion paradigms. The area of each bubble represents the memory usage of the corresponding model. RED demonstrates superior performance while maintaining both low inference time and low memory usage.
• Figure 2: Workflow of the proposed RED model. The illustration depicts the reversible diffusion process and the knowledge-guidance module. The backbone adopts a U-Net architecture, where the standard residual blocks are replaced by reversible residual blocks.
• Figure 3: Qualitative comparisons of SOTA methods in the LLVIP, MSRS, M$^3$FD datasets.
• Figure 4: Qualitative comparisons of SOTA methods in Harvard Medical dataset.
• Figure 5: Visualization of representative details lost exhibited by Text-DiFuse.