Generative Adversarial Networks for Spatio-Spectral Compression of Hyperspectral Images

TL;DR

This work addresses the challenge of efficiently compressing hyperspectral images by exploiting spatio-spectral redundancies. It extends the HiFiC GAN-based spatial compression framework to HSIs through two models: HiFiC$_{SE}$, which uses SE channel-attention, and HiFiC$_{3D}$, which incorporates 3D convolutions to capture inter-band dependencies, both within a rate-distortion adversarial objective that includes a hyper-prior. The study demonstrates that these spatio-spectral variants achieve lower bitrates and higher perceptual reconstruction quality than JPEG 2000 at low bitrates, with HiFiC$_{3D}$ often providing the best performance across most configurations. The results underscore the practical potential of GAN-based spatio-spectral compression for remote sensing data, and the public code enables further adoption and benchmarking in the field.

Abstract

The development of deep learning-based models for the compression of hyperspectral images (HSIs) has recently attracted great attention in remote sensing due to the sharp growing of hyperspectral data archives. Most of the existing models achieve either spectral or spatial compression, and do not jointly consider the spatio-spectral redundancies present in HSIs. To address this problem, in this paper we focus our attention on the High Fidelity Compression (HiFiC) model (which is proven to be highly effective for spatial compression problems) and adapt it to perform spatio-spectral compression of HSIs. In detail, we introduce two new models: i) HiFiC using Squeeze and Excitation (SE) blocks (denoted as HiFiC$_{SE}$); and ii) HiFiC with 3D convolutions (denoted as HiFiC$_{3D}$) in the framework of compression of HSIs. We analyze the effectiveness of HiFiC$_{SE}$ and HiFiC$_{3D}$ in compressing the spatio-spectral redundancies with channel attention and inter-dependency analysis. Experimental results show the efficacy of the proposed models in performing spatio-spectral compression, while reconstructing images at reduced bitrates with higher reconstruction quality. The code of the proposed models is publicly available at https://git.tu-berlin.de/rsim/HSI-SSC .

Figures (5)

• Figure 1: \ref{['fig:architecture']} GAN-based HiFiC model mentzer2020highfidelity. \ref{['fig:e']}, \ref{['fig:g']} and \ref{['fig:d']} show the Encoder, Generator and Discriminator architectures of HiFiC$_{opt}$, HiFiC$_{SE}$ with added SE Blocks and HiFiC$_{3D}$ with two Conv3D layers, respectively. Q: quantization, P: probability model, AE: arithmetic encoder, AD: arithmetic decoder, Conv3D(T)-N-KxK(xK): 2D/3D (transposed) convolution layer with N output channels and kernel size K, $\downarrow^2$/$\uparrow_2$: stride 2, CNorm: channel normalization layer, NN-$\uparrow_{16}$: nearest neighbor upsampling.
• Figure 2: The basic structure of an SE block squeeze.
• Figure 3: (a) Original image; (b) reconstructed image without 3D convs; and (c) reconstructed image with 3D convs.
• Figure 4: Test set rate-distortion performance of our proposed HiFiC$_{SE}$ and HiFiC$_{3D}$ compared to HiFiC$_{opt}$ and JPEG2000.
• Figure 5: (a) Original. Reconstructed with (b) HiFiC$_{opt}$, (c) HiFiC$_{SE}$ and (d) HiFiC$_{3D}$. The PSNR and achieved bpp are shown below each image as (PSNR in dB, bpp).