Synthetic Abundance Maps for Unsupervised Super-Resolution of Hyperspectral Remote Sensing Images

TL;DR

This work tackles the lack of ground-truth data for hyperspectral single-image super-resolution by introducing an unsupervised framework that trains on synthetic abundance maps. It unmixes a low-resolution HSI into endmembers and abundances, then uses a dead leaves model to generate realistic high- and low-resolution abundance pairs to supervise a super-resolution network, finally reconstructing the high-resolution HSI from endmembers. Key contributions include the dead leaves-based synthetic data generation tailored to remote sensing, integration with hyperspectral unmixing, and demonstrations of robustness across urban scene datasets, outperforming several supervised baselines at higher scale factors. The approach offers flexible data generation and network choices, reducing reliance on ground-truth high-resolution hyperspectral data and enabling robust SR in practical remote sensing applications, with potential extensions to PSF estimation and more general synthetic data design.

Abstract

Hyperspectral single image super-resolution (HS-SISR) aims to enhance the spatial resolution of hyperspectral images to fully exploit their spectral information. While considerable progress has been made in this field, most existing methods are supervised and require ground truth data for training-data that is often unavailable in practice. To overcome this limitation, we propose a novel unsupervised training framework for HS-SISR, based on synthetic abundance data. The approach begins by unmixing the hyperspectral image into endmembers and abundances. A neural network is then trained to perform abundance super-resolution using synthetic abundances only. These synthetic abundance maps are generated from a dead leaves model whose characteristics are inherited from the low-resolution image to be super-resolved. This trained network is subsequently used to enhance the spatial resolution of the original image's abundances, and the final super-resolution hyperspectral image is reconstructed by combining them with the endmembers. Experimental results demonstrate both the training value of the synthetic data and the effectiveness of the proposed method.

Figures (8)

• Figure 1: Structure of the proposed method: The super-resolution network is trained with synthetic pairs $(A_{DL,SR},A_{DL,LR})$ and then used to super-resolve $A_{LR}$ into $A_{SR}$.
• Figure 2: Illustration of synthetic abundance generation for $A_{DL,HR}$ using the $1^{\text{st}}$, $2^{\text{nd}}$, and $3^{\text{rd}}$ value columns of $v_{\text{extract}}$ in (a), (b), and (c), respectively. In (c), an example highlights the occlusion mechanism: a newly added leaf is partially covered by a previously generated one, illustrating the stacking behavior of the dead leaves model.
• Figure 3: Comparison between real urban abundance (Top) and synthetic dead leaves abundance (Bottom).
• Figure 4: Visual comparison at bands 20, 50, and 100 of a Chikusei patch with a $\times 2$ scaling factor: high-resolution reference (HR), low-resolution (LR), and super-resolved (SR) images generated using bicubic interpolation, MCNet, SSPSR, HSISR, and MCNet-DL.
• Figure 5: Visual comparison at bands 20, 50, and 100 of a Pavia University patch with a $\times 3$ scaling factor: high-resolution reference (HR), low-resolution (LR), and super-resolved (SR) images generated using bicubic interpolation, MCNet, SSPSR, HSISR, and MCNet-DL.