UnmixingSR: Material-aware Network with Unsupervised Unmixing as Auxiliary Task for Hyperspectral Image Super-resolution

TL;DR

UnmixingSR tackles hyperspectral image super-resolution by integrating unsupervised hyperspectral unmixing as an auxiliary task to exploit material component information. The method combines a primary SR network with a shared, unsupervised unmixing autoencoder and introduces a material-aware module (MAM) built on GRAM blocks to enforce LR–HR abundance consistency through a hybrid loss framework. Key contributions include the design of an end-to-end, plug-in HU auxiliary task, the GRAM-based feature extraction that supports both SR and unmixing, and empirical demonstrations of state-of-the-art performance on multiple datasets with improved spectral and spatial fidelity. The approach yields physically meaningful SR results and offers a robust, adaptable augmentation that can be embedded into existing HSI SR architectures with limited extra data requirements.

Abstract

Deep learning-based (DL-based) hyperspectral image (HIS) super-resolution (SR) methods have achieved remarkable performance and attracted attention in industry and academia. Nonetheless, most current methods explored and learned the mapping relationship between low-resolution (LR) and high-resolution (HR) HSIs, leading to the side effect of increasing unreliability and irrationality in solving the ill-posed SR problem. We find, quite interestingly, LR imaging is similar to the mixed pixel phenomenon. A single photodetector in sensor arrays receives the reflectance signals reflected by a number of classes, resulting in low spatial resolution and mixed pixel problems. Inspired by this observation, this paper proposes a component-aware HSI SR network called UnmixingSR, in which the unsupervised HU as an auxiliary task is used to perceive the material components of HSIs. We regard HU as an auxiliary task and incorporate it into the HSI SR process by exploring the constraints between LR and HR abundances. Instead of only learning the mapping relationship between LR and HR HSIs, we leverage the bond between LR abundances and HR abundances to boost the stability of our method in solving SR problems. Moreover, the proposed unmixing process can be embedded into existing deep SR models as a plug-in-play auxiliary task. Experimental results on hyperspectral experiments show that unmixing process as an auxiliary task incorporated into the SR problem is feasible and rational, achieving outstanding performance. The code is available at

Figures (9)

• Figure 1: The illustration of the relationship between the LR HSIs and the mixed pixels.
• Figure 2: The relationship between the LR-HR abundances.
• Figure 3: Overview of the UnmixingSR. The proposed framework mainly contains two parts: the primary SR network and the auxiliary HU network. We build the relationship between the primary and auxiliary task by the material-aware module. The auxiliary unmixing network shares the weight. Specifically, the weight of the decoder can be regarded as the endmembers.
• Figure 4: The architecture of the unmixing framework.
• Figure 5: The architecture of the GRAM framework.