Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hyperspectral Super-Resolution with Inter-Image Variability via Degradation-based Low-Rank and Residual Fusion Method

Yue Wen, Kunjing Yang, Minru Bai

TL;DR

This work tackles hyperspectral super-resolution by fusing a low-resolution HSI with a high-resolution MSI in the presence of inter-image variability. The authors model spectral variability as changes in the spectral degradation operator $\mathbf{P}_3 = \mathbf{R} + \Delta \mathbf{R}$ and recover spatial details by decomposing the target HSI into a low-rank part and a residual part, expressed in a subspace representation to reduce dimensionality. An implicit regularizer, implemented via a Plug-and-Play denoiser, enforces spatial priors within a Proximal Alternating Optimization framework, and a rigorous convergence analysis based on the Kurdyka–Łojasiewicz property is provided. Extensive experiments on four HSI/MSI pairs with inter-image variability show that DLRRF achieves superior fusion performance across multiple metrics and improves downstream classification, highlighting its practical impact for robust HSR under real-world acquisition conditions.

Abstract

The fusion of hyperspectral image (HSI) with multispectral image (MSI) provides an effective way to enhance the spatial resolution of HSI. However, due to different acquisition conditions, there may exist spectral variability and spatially localized changes between HSI and MSI, referred to as inter-image variability, which can significantly affect the fusion performance. Existing methods typically handle inter-image variability by applying direct transformations to the images themselves, which can exacerbate the ill-posedness of the fusion model. To address this challenge, we propose a Degradation-based Low-Rank and Residual Fusion (DLRRF) model. First, we model the spectral variability as change in the spectral degradation operator. Second, to recover the lost spatial details caused by spatially localized changes, we decompose the target HSI into low rank and residual components, where the latter is used to capture the lost details. By exploiting the spectral correlation within the images, we perform dimensionality reduction on both components. Additionally, we introduce an implicit regularizer to utilize the spatial prior information from the images. The proposed DLRRF model is solved using the Proximal Alternating Optimization (PAO) algorithm within a Plug-and-Play (PnP) framework, where the subproblem regarding implicit regularizer is addressed by an external denoiser. We further provide a comprehensive convergence analysis of the algorithm. Finally, extensive numerical experiments demonstrate that DLRRF achieves superior performance in fusing HSI and MSI with inter-image variability.

Hyperspectral Super-Resolution with Inter-Image Variability via Degradation-based Low-Rank and Residual Fusion Method

TL;DR

This work tackles hyperspectral super-resolution by fusing a low-resolution HSI with a high-resolution MSI in the presence of inter-image variability. The authors model spectral variability as changes in the spectral degradation operator and recover spatial details by decomposing the target HSI into a low-rank part and a residual part, expressed in a subspace representation to reduce dimensionality. An implicit regularizer, implemented via a Plug-and-Play denoiser, enforces spatial priors within a Proximal Alternating Optimization framework, and a rigorous convergence analysis based on the Kurdyka–Łojasiewicz property is provided. Extensive experiments on four HSI/MSI pairs with inter-image variability show that DLRRF achieves superior fusion performance across multiple metrics and improves downstream classification, highlighting its practical impact for robust HSR under real-world acquisition conditions.

Abstract

The fusion of hyperspectral image (HSI) with multispectral image (MSI) provides an effective way to enhance the spatial resolution of HSI. However, due to different acquisition conditions, there may exist spectral variability and spatially localized changes between HSI and MSI, referred to as inter-image variability, which can significantly affect the fusion performance. Existing methods typically handle inter-image variability by applying direct transformations to the images themselves, which can exacerbate the ill-posedness of the fusion model. To address this challenge, we propose a Degradation-based Low-Rank and Residual Fusion (DLRRF) model. First, we model the spectral variability as change in the spectral degradation operator. Second, to recover the lost spatial details caused by spatially localized changes, we decompose the target HSI into low rank and residual components, where the latter is used to capture the lost details. By exploiting the spectral correlation within the images, we perform dimensionality reduction on both components. Additionally, we introduce an implicit regularizer to utilize the spatial prior information from the images. The proposed DLRRF model is solved using the Proximal Alternating Optimization (PAO) algorithm within a Plug-and-Play (PnP) framework, where the subproblem regarding implicit regularizer is addressed by an external denoiser. We further provide a comprehensive convergence analysis of the algorithm. Finally, extensive numerical experiments demonstrate that DLRRF achieves superior performance in fusing HSI and MSI with inter-image variability.

Paper Structure

This paper contains 21 sections, 4 theorems, 70 equations, 6 figures, 7 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. PRELIMINARIES AND PROBLEM FORMULATION
  3. Notation and Preliminaries
  4. Problem Formulation
  5. THE PROPOSED DLRRF MODEL
  6. THE PAO ALGORITHM FOR SOLVING DLRRF MODEL
  7. Optimization with respect to $\mathcal{L}$
  8. Optimization with respect to $\mathcal{E}$
  9. Optimization with respect to $\bigtriangleup \mathbf{R}$
  10. CONVERGENCE ANALYSIS
  11. EXPERIMENTAL RESULTS
  12. Experimental Datasets
  13. Experimental Settings
  14. Evaluation Indexes
  15. Experimental Results
  16. ...and 6 more sections

Key Result

Lemma 1

Let $\{\mathcal{W}^k = (\mathcal{L}^k, \mathcal{E}^k, \Delta \mathbf{R}^k) \}_{k \in \mathbb{N}}$ be a sequence generated by the PAO algorithm. Then the following conclusions hold: hence $\lim_{k \to +\infty} \|\mathcal{W}^{k+1} - \mathcal{W}^k\| = 0$.

Figures (6)

  • Figure 1: Hyperspectral and multispectral images used in the experiments.
  • Figure 2: The first and second rows show the visible and infrared representation for the reconstructed images of the Isabella Lake dataset when $sf=2$, respectively.
  • Figure 3: The first and second rows show the visible and infrared representation for the reconstructed images of the Lake Tahoe A dataset when $sf=4$, respectively.
  • Figure 4: The first and second rows show the visible and infrared representation for the reconstructed images of the Lake Tahoe B dataset when $sf=2$, respectively.
  • Figure 5: The first and second rows show the visible and infrared representation for the reconstructed images of the Ivanpah Playa dataset when $sf=4$, respectively.
  • ...and 1 more figures

Theorems & Definitions (6)

  • Remark 3.1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Theorem 1
  • Remark 5.1: Stopping Criterion