Hipandas: Hyperspectral Image Joint Denoising and Super-Resolution by Image Fusion with the Panchromatic Image

TL;DR

This work tackles the practical problem of jointly denoising and super-resolving hyperspectral images by fusing high-resolution PAN data in a zero-shot setting. It introduces ZSHipandas, a three-network architecture (GDN, GSRN, PRN) augmented with a detail-oriented low-rank prior and a two-stage training procedure to mitigate training bias and leverage cross-modal guidance. The method delivers state-of-the-art performance on both simulated and real-world datasets, with quantitative gains in PSNR/SSIM and qualitative improvements in texture and color fidelity, while ablations confirm the necessity of each component and the PAN-HS fusion. The approach holds significant practical impact for satellite imaging, enabling more accurate, high-resolution hyperspectral imagery in scenarios with noisy, downsampled data.

Abstract

Hyperspectral images (HSIs) are frequently noisy and of low resolution due to the constraints of imaging devices. Recently launched satellites can concurrently acquire HSIs and panchromatic (PAN) images, enabling the restoration of HSIs to generate clean and high-resolution imagery through fusing PAN images for denoising and super-resolution. However, previous studies treated these two tasks as independent processes, resulting in accumulated errors. This paper introduces \textbf{H}yperspectral \textbf{I}mage Joint \textbf{Pand}enoising \textbf{a}nd Pan\textbf{s}harpening (Hipandas), a novel learning paradigm that reconstructs HRHS images from noisy low-resolution HSIs (LRHS) and high-resolution PAN images. The proposed zero-shot Hipandas framework consists of a guided denoising network, a guided super-resolution network, and a PAN reconstruction network, utilizing an HSI low-rank prior and a newly introduced detail-oriented low-rank prior. The interconnection of these networks complicates the training process, necessitating a two-stage training strategy to ensure effective training. Experimental results on both simulated and real-world datasets indicate that the proposed method surpasses state-of-the-art algorithms, yielding more accurate and visually pleasing HRHS images.

Figures (7)

• Figure 1: The differences among (a) pansharpening, (b) pandenoising and (c) Hipandas. Pansharpening addresses super-resolution, while pandenoising focuses on denoising. In contrast, Hipandas is specifically designed to tackle both problems in a unified framework.
• Figure 2: The framework for the proposed ZSHipandas. It is pretrained in stage 1 on LR scale images, and then finetuned in stage 2 on HR scale images. The GDN and GSRN components are used for image enhancement, while the PRN component models relationship between HS and PAN images.
• Figure 3: The energy curve of clean and corrupted detail maps (i.e., $D$ and $\tilde{D}$). $\sigma$ and $p$ denote the noise intensity for Gaussian and mixture noise, respectively.
• Figure 4: The architecture for the (a) GDN/GSRN component, (b) PRN component and (c) HPF layer. Conv and SConv denote convolutional and strided convolutional units, respectively. AAP denotes the adaptive average pooling.
• Figure 5: Restoration results for Gaussian noise with $\sigma=30$.