Deep Diversity-Enhanced Feature Representation of Hyperspectral Images

TL;DR

This work introduces a rank-based perspective on feature diversity for hyperspectral image analysis and proposes ReS$^3$-ConvSet, a rank-enhanced, symmetry-preserving convolution module. To further promote diversity, a content-level regularizer DA-Reg encourages a more uniform distribution of feature map singular values. The approach is integrated into HS denoising, spatial SR, and classification, with extensive experiments showing state-of-the-art or near-state-of-the-art results while using fewer parameters than many baselines. The combination of structure-level and content-level diversity enhancements provides a unified framework for efficient, diverse HS feature learning with broad practical impact.

Abstract

In this paper, we study the problem of efficiently and effectively embedding the high-dimensional spatio-spectral information of hyperspectral (HS) images, guided by feature diversity. Specifically, based on the theoretical formulation that feature diversity is correlated with the rank of the unfolded kernel matrix, we rectify 3D convolution by modifying its topology to enhance the rank upper-bound. This modification yields a rank-enhanced spatial-spectral symmetrical convolution set (ReS$^3$-ConvSet), which not only learns diverse and powerful feature representations but also saves network parameters. Additionally, we also propose a novel diversity-aware regularization (DA-Reg) term that directly acts on the feature maps to maximize independence among elements. To demonstrate the superiority of the proposed ReS$^3$-ConvSet and DA-Reg, we apply them to various HS image processing and analysis tasks, including denoising, spatial super-resolution, and classification. Extensive experiments show that the proposed approaches outperform state-of-the-art methods both quantitatively and qualitatively to a significant extent. The code is publicly available at https://github.com/jinnh/ReSSS-ConvSet.

Figures (12)

• Figure 1: Illustration of re-permuting the 3-D convolutional kernel set (a) into rank-enhanced spatial-spectral symmetrical patterns with different rank upper-bounds in (b), (c), and (d). Specifically, The colored cubes symbolize the kernel weights, whereas the blank ones represent replenished zeros. (a) the unrolled 3-D convolutional kernel matrix with the kernel size of 3. Meanwhile, (b), (c), and (d) represent to re-permute 3-D convolutional kernel matrix in our rank-enhanced spatial-spectral symmetrical manner with the kernel size of $3\times 3$, $1\times 3$, and $1\times 3$, of which the scaling factors are $3$, $9$, and $3$, individually.
• Figure 2: Illustration of various feature extraction manners for HS images. (a) 3-D convolution, (b) Sequential 1-D and 2-D convolution, (c) Sequential 1-D convolution, (d) 1-D + 2-D convolution, and (e) Proposed ReS$^3$-ConvSet.
• Figure 3: Comparison of the distribution of the singular values of the feature maps extracted by different convolution manners used in the applications of (a) HS image denoising on the ICVL dataset with the Gaussian noise ($\sigma=70$), (b) HS image $4\times$ spatial super-resolution on the CAVE dataset, and (c) HS image classification on Indian Pines. The singular values of each convolution scheme are normalized by its largest value.
• Figure 4: Illustration of our HS image denoising framework, which is constructed by incorporating the proposed ReS$^3$-ConvSet into a residual U-Net architecture.
• Figure 5: Illustration of our HS image super-resolution framework, which is constructed by incorporating the proposed ReS$^3$-ConvSet into a residual-dense architecture.