A Hybrid Registration and Fusion Method for Hyperspectral Super-resolution

TL;DR

This work tackles the challenge of fusing hyperspectral and multispectral images when misregistration and diverse sensor characteristics hinder performance. It introduces a two-stage RAF-NLRGS framework: first, RAF performs batch image alignment and fusion within a subspace-regularized objective, solved via Generalized Gauss–Newton and symmetric Gauss–Seidel ADMM; second, NLRGS refines the fusion by modeling a primary low-rank component plus a group-sparse residual in separate spectral subspaces, solved with Proximal Alternating Optimization. The authors establish error bounds for NLRGS, prove convergence properties of the algorithms, and demonstrate superior fusion quality and robustness across synthetic misalignment scenarios and real-world GF1-GF5 data, outperforming several state-of-the-art methods. The approach offers a practical, theoretically grounded path to reliable HSI-MSI fusion in challenging real-world settings, with potential impact on material identification and remote sensing analyses.

Abstract

Fusing hyperspectral images (HSIs) with multispectral images (MSIs) has become a mainstream approach to enhance the spatial resolution of HSIs. Many HSI-MSI fusion methods have achieved impressive results. Nevertheless, certain challenges persist, including: (a) A majority of current methods rely on accurate registration of HSI and MSI, which can be challenging in real-world applications.(b) The obtained HSI-MSI pairs may not be fully utilized. In this paper, we propose a hybrid registration and fusion constrained optimization model named RAF-NLRGS. With respect to challenge (a), the RAF model integrates batch image alignment within the fusion process, facilitating simultaneous execution of image registration and fusion. To address issue (b), the NLRGS model incorporates a nonconvex low-rank and group-sparse structure, leveraging group sparsity to effectively harness valuable information embedded in the residual data. Moreover, the NLRGS model can further enhance fusion performance based on the RAF model. Subsequently, the RAF-NLRGS model is solved within the framework of Generalized Gauss-Newton (GGN) algorithm and Proximal Alternating Optimization (PAO) algorithm. Theoretically, we establish the error bounds for the NLRGS model and the convergence analysis of corresponding algorithms is also presented. Finally, extensive numerical experiments on HSI datasets are conducted to verify the effectiveness of our method.

Figures (9)

• Figure 1: (a) Unregistered image pairs: $\mathcal{Y}$ and $\mathcal{Z}$. (b) Registered image pairs: $\mathcal{Y}_{R}$, $\mathcal{Z}$ and preliminary fused image $\mathcal{X}_1$. (c) Final fused image $\mathcal{X}$.
• Figure 2: Visualization of the registration and fusion efficacy for the RAF-NLRGS model (pseudocolor image, bands 20, 40, 60). (a) True HSI. (h) MSI. (b)-(f) Transformed HSIs. (i)-(m) The corresponding registrated HSIs. (g) The fused HSI. (n) The error image.
• Figure 3: The first row presents the reconstructed images of the compared methods for Pavia University. The second row shows corresponding error images.
• Figure 4: The correlation between the image restoration quality of different methodologies and the scale of transformation on 'Pavia University'.
• Figure 5: The input original GF1-GF5 image pair, along with the fused images (bands 30, 60, and 90) resulting from compared methodologies.

Theorems & Definitions (16)

• Definition 3.1
• Definition 3.2
• Definition 3.3
• Definition 3.4
• Definition 3.5
• Definition 3.6
• Theorem 4.1
• Remark 4.1
• Theorem 4.2
• Theorem 4.3