Rethinking the Upsampling Layer in Hyperspectral Image Super Resolution

TL;DR

SHSR faces high spectral dimensionality and limited real-time capability. The authors introduce LKCA-Net, a lightweight SHSR framework built on Large-Kernel Channel Attention and a learnable upsampling layer, and reveal that the upsampling layer is a key bottleneck due to low-rank structure. They address this bottleneck with a low-rank upsampling approximation, plus a knowledge-distillation–based feature alignment to preserve representation quality; a KD loss with spectral and gradient constraints is combined with a decaying teacher influence during training. Across Chikusei, Houston 2018, and Pavia Center, LKCA-Net achieves competitive accuracy with orders-of-magnitude speedups, maintaining performance via LKCA-LR and LKCA-KD variants, and demonstrating practical applicability in resource-constrained settings.

Abstract

Deep learning has achieved significant success in single hyperspectral image super-resolution (SHSR); however, the high spectral dimensionality leads to a heavy computational burden, thus making it difficult to deploy in real-time scenarios. To address this issue, this paper proposes a novel lightweight SHSR network, i.e., LKCA-Net, that incorporates channel attention to calibrate multi-scale channel features of hyperspectral images. Furthermore, we demonstrate, for the first time, that the low-rank property of the learnable upsampling layer is a key bottleneck in lightweight SHSR methods. To address this, we employ the low-rank approximation strategy to optimize the parameter redundancy of the learnable upsampling layer. Additionally, we introduce a knowledge distillation-based feature alignment technique to ensure the low-rank approximated network retains the same feature representation capacity as the original. We conducted extensive experiments on the Chikusei, Houston 2018, and Pavia Center datasets compared to some SOTAs. The results demonstrate that our method is competitive in performance while achieving speedups of several dozen to even hundreds of times compared to other well-performing SHSR methods.

Figures (8)

• Figure 1: Illustration of the number of parameters for different functional layers of VDSR vdsr and ESSAfomer essaformer. Blue represents the proportion of the learnable upsampling layer's parameters, while gray represents the backbone.
• Figure 2: Overall Architecture of LKCA-Net
• Figure 3: Large-Kernel Channel Attention-based Block
• Figure 4: Cumulative Sum of Normalized Singular Values.
• Figure 5: The original $3\times3$ convolution is approximated by a group convolution with $g$ groups, reducing the number of parameters by $g$. The low-rank matrix $M_l$ is formed by reshaping the parameters of the group convolution.