Self-Learning Hyperspectral and Multispectral Image Fusion via Adaptive Residual Guided Subspace Diffusion Model

TL;DR

This work tackles HSI-MSI fusion under limited training data by introducing ARGS-Diff, a self-learning diffusion-based framework that decomposes HR-HSI into a spectral basis $\mathbf{E}$ and a reduced coefficient $\mathcal{A}$. It learns two lightweight networks (spectral and spatial) from LR-HSI and HR-MSI and uses a diffusion reverse process to reconstruct the HR-HSI from the subspace components, guided by the observed data. A novel Adaptive Residual Guided Module (ARGM) stabilizes the joint diffusion updates, leading to improved convergence and robustness. Across multiple simulated and real datasets, ARGS-Diff achieves state-of-the-art fusion accuracy with substantially lower computational cost and memory usage, demonstrating strong practical potential for remote sensing applications.

Abstract

Hyperspectral and multispectral image (HSI-MSI) fusion involves combining a low-resolution hyperspectral image (LR-HSI) with a high-resolution multispectral image (HR-MSI) to generate a high-resolution hyperspectral image (HR-HSI). Most deep learning-based methods for HSI-MSI fusion rely on large amounts of hyperspectral data for supervised training, which is often scarce in practical applications. In this paper, we propose a self-learning Adaptive Residual Guided Subspace Diffusion Model (ARGS-Diff), which only utilizes the observed images without any extra training data. Specifically, as the LR-HSI contains spectral information and the HR-MSI contains spatial information, we design two lightweight spectral and spatial diffusion models to separately learn the spectral and spatial distributions from them. Then, we use these two models to reconstruct HR-HSI from two low-dimensional components, i.e, the spectral basis and the reduced coefficient, during the reverse diffusion process. Furthermore, we introduce an Adaptive Residual Guided Module (ARGM), which refines the two components through a residual guided function at each sampling step, thereby stabilizing the sampling process. Extensive experimental results demonstrate that ARGS-Diff outperforms existing state-of-the-art methods in terms of both performance and computational efficiency in the field of HSI-MSI fusion. Code is available at https://github.com/Zhu1116/ARGS-Diff.

Figures (6)

• Figure 1: The overall framework of the proposed ARGS-Diff.
• Figure 2: Visual results and reconstruction error maps obtained by different methods. The first, third, and fifth rows are the pseudo-color images of Pavia, Chikusei, and KSC datasets, and the second, fourth, and sixth rows correspond to their error maps.
• Figure 3: Visual results obtained by different methods on the Houston dataset.
• Figure 4: Intermediate results of the reduced coefficient $\mathcal{A}$. The first and second rows represent 'w/o ARGM' and 'w/ ARGM'.
• Figure 5: Sensitivity analysis of the parameters $d$ and $T$.