Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SHARE: A Fully Unsupervised Framework for Single Hyperspectral Image Restoration

Jiangwei Xie, Zhang Wen, Mike Davies, Dongdong Chen

TL;DR

SHARE addresses the critical challenge of hyperspectral image restoration without ground-truth data by combining transformation-based equivariant learning with a robust SURE loss and a memory-augmented, low-rank spectral attention module (DASA). The method maps a single degraded HSI via an inverse network and enforces consistency through measurement and robust equivariance constraints, enabling accurate inpainting and super-resolution in a fully unsupervised, zero-shot setting. Key contributions include the formulation of robust equivariance constraints, the SURE-based loss for noise resilience, and the Dynamic Adaptive Spectral Attention mechanism that leverages global low-rank spectral structure. Empirical results on multiple datasets show that SHARE outperforms prior unsupervised approaches and achieves performance comparable to supervised methods, highlighting its potential for real-world scientific imaging where ground-truth data are scarce.

Abstract

Hyperspectral image (HSI) restoration is a fundamental challenge in computational imaging and computer vision. It involves ill-posed inverse problems, such as inpainting and super-resolution. Although deep learning methods have transformed the field through data-driven learning, their effectiveness hinges on access to meticulously curated ground-truth datasets. This fundamentally restricts their applicability in real-world scenarios where such data is unavailable. This paper presents SHARE (Single Hyperspectral Image Restoration with Equivariance), a fully unsupervised framework that unifies geometric equivariance principles with low-rank spectral modelling to eliminate the need for ground truth. SHARE's core concept is to exploit the intrinsic invariance of hyperspectral structures under differentiable geometric transformations (e.g. rotations and scaling) to derive self-supervision signals through equivariance consistency constraints. Our novel Dynamic Adaptive Spectral Attention (DASA) module further enhances this paradigm shift by explicitly encoding the global low-rank property of HSI and adaptively refining local spectral-spatial correlations through learnable attention mechanisms. Extensive experiments on HSI inpainting and super-resolution tasks demonstrate the effectiveness of SHARE. Our method outperforms many state-of-the-art unsupervised approaches and achieves performance comparable to that of supervised methods. We hope that our approach will shed new light on HSI restoration and broader scientific imaging scenarios. The code will be released at https://github.com/xuwayyy/SHARE.

SHARE: A Fully Unsupervised Framework for Single Hyperspectral Image Restoration

TL;DR

SHARE addresses the critical challenge of hyperspectral image restoration without ground-truth data by combining transformation-based equivariant learning with a robust SURE loss and a memory-augmented, low-rank spectral attention module (DASA). The method maps a single degraded HSI via an inverse network and enforces consistency through measurement and robust equivariance constraints, enabling accurate inpainting and super-resolution in a fully unsupervised, zero-shot setting. Key contributions include the formulation of robust equivariance constraints, the SURE-based loss for noise resilience, and the Dynamic Adaptive Spectral Attention mechanism that leverages global low-rank spectral structure. Empirical results on multiple datasets show that SHARE outperforms prior unsupervised approaches and achieves performance comparable to supervised methods, highlighting its potential for real-world scientific imaging where ground-truth data are scarce.

Abstract

Hyperspectral image (HSI) restoration is a fundamental challenge in computational imaging and computer vision. It involves ill-posed inverse problems, such as inpainting and super-resolution. Although deep learning methods have transformed the field through data-driven learning, their effectiveness hinges on access to meticulously curated ground-truth datasets. This fundamentally restricts their applicability in real-world scenarios where such data is unavailable. This paper presents SHARE (Single Hyperspectral Image Restoration with Equivariance), a fully unsupervised framework that unifies geometric equivariance principles with low-rank spectral modelling to eliminate the need for ground truth. SHARE's core concept is to exploit the intrinsic invariance of hyperspectral structures under differentiable geometric transformations (e.g. rotations and scaling) to derive self-supervision signals through equivariance consistency constraints. Our novel Dynamic Adaptive Spectral Attention (DASA) module further enhances this paradigm shift by explicitly encoding the global low-rank property of HSI and adaptively refining local spectral-spatial correlations through learnable attention mechanisms. Extensive experiments on HSI inpainting and super-resolution tasks demonstrate the effectiveness of SHARE. Our method outperforms many state-of-the-art unsupervised approaches and achieves performance comparable to that of supervised methods. We hope that our approach will shed new light on HSI restoration and broader scientific imaging scenarios. The code will be released at https://github.com/xuwayyy/SHARE.
Paper Structure (34 sections, 11 equations, 28 figures, 8 tables)

This paper contains 34 sections, 11 equations, 28 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. HSI Restoration
  4. Equivariant Imaging
  5. Method
  6. Problem Statement
  7. Single HSI Restoration with Equivariance
  8. Measurement Consistency
  9. Equivariance Consistency
  10. Robust Equivariance Constraint
  11. SURE Loss
  12. Inverse Mapping $f_\theta$
  13. Overview of $f_\theta$
  14. Dynamic Adaptive Spectral Attention
  15. Loss Function
  16. ...and 19 more sections

Figures (28)

  • Figure 1: An illustration of learning with equivariance in the HSI inpainting task. The reconstructed image after applying a transformation should be equivalent to the restoration of the initially transformed and masked image.
  • Figure 2: The overall pipeline of SHARE. Given only one noisy observation $y$, it is first passed through the inverse mapping $f_\theta$ to obtain a restored estimate $f_\theta(y)$. This estimate is then forwarded through the physics model $\mathcal{H}$ to compute $\mathcal{H}(f_\theta(y))$, which is compared with the observation $y$ to calculate the SURE loss. Simultaneously, $f_\theta(y)$ undergoes a transformation $T_g$, followed by the physics model, noise corruption, and inverse mapping, to compute the robust equivariance consistency loss.
  • Figure 3: (a) The overall architecture of the inverse mapping $f_\theta$, which adopts a U-shaped design. (b) The structure of the convolutional block. It doubles the input's channel dimension in the encoding phase and halves it in the decoding phase. (c) The architecture of the decoder block consists of a DASA module, a convolutional block, and a transposed convolution layer. (d) The internal structure of the proposed DASA module.
  • Figure 4: Inpainting results of different methods on Chikusei dataset. MPSNR values are shown on the top left, MSSIM and SAM values are shown below each image. The best and second-best values are marked in red and blue, respectively. Band 90 for visualization. Please zoom in for a better view.
  • Figure 5: Inpainting results of different methods on Indian-Pines dataset. MPSNR values are shown on the top left, MSSIM and SAM values are shown below each image. The best and second-best values are marked in red and blue, respectively. Band 149 for visualization. Please zoom in for a better view.
  • ...and 23 more figures