Deep Equilibrium Convolutional Sparse Coding for Hyperspectral Image Denoising

TL;DR

<3-5 sentence high-level summary> This paper tackles hyperspectral image denoising under complex noise by introducing a Deep Equilibrium Convolutional Sparse Coding (DECSC) framework. DECSC decomposes the HSI into global inter-band common and local spatial-spectral unique components, enforcing shared sparse codes across bands and augmenting nonlocal similarity with a Swin Transformer plus a detail enhancement module for fine details. The optimization is reformulated as a fixed-point problem and solved with a DEQ layer using Anderson acceleration and phantom gradients, enabling infinite-depth, memory-efficient training with convergence guarantees. Empirical results on synthetic and real-world HSIs demonstrate state-of-the-art denoising performance and robustness to challenging noise patterns.

Abstract

Hyperspectral images (HSIs) play a crucial role in remote sensing but are often degraded by complex noise patterns. Ensuring the physical property of the denoised HSIs is vital for robust HSI denoising, giving the rise of deep unfolding-based methods. However, these methods map the optimization of a physical model to a learnable network with a predefined depth, which lacks convergence guarantees. In contrast, Deep Equilibrium (DEQ) models treat the hidden layers of deep networks as the solution to a fixed-point problem and models them as infinite-depth networks, naturally consistent with the optimization. Under the framework of DEQ, we propose a Deep Equilibrium Convolutional Sparse Coding (DECSC) framework that unifies local spatial-spectral correlations, nonlocal spatial self-similarities, and global spatial consistency for robust HSI denoising. Within the convolutional sparse coding (CSC) framework, we enforce shared 2D convolutional sparse representation to ensure global spatial consistency across bands, while unshared 3D convolutional sparse representation captures local spatial-spectral details. To further exploit nonlocal self-similarities, a transformer block is embedded after the 2D CSC. Additionally, a detail enhancement module is integrated with the 3D CSC to promote image detail preservation. We formulate the proximal gradient descent of the CSC model as a fixed-point problem and transform the iterative updates into a learnable network architecture within the framework of DEQ. Experimental results demonstrate that our DECSC method achieves superior denoising performance compared to state-of-the-art methods.

Figures (15)

• Figure 1: DEQ-based inference of the proposed model. Forward pass uses Anderson acceleration; backward pass uses an unrolling-based phantom gradient.
• Figure 2: Illustration of the DECSC architecture. The network layer $f_{\Theta}$ integrates the ISTA backbone, a Swin Transformer for capturing non-local dependencies, and a difference convolution module with an attention mechanism for enhancing fine details. The fixed point is directly solved using Anderson acceleration
• Figure 3: The difference convolution module consists of a 3D convolution and four HSI-specific edge extraction operators.
• Figure 4: Denoising results on the $Nachal\_0823-1214$ HSI from the ICVL dataset under the non-i.i.d. Gaussian noise with $\sigma \in [0,95]$. The false-color images are generated by combining bands 31, 17, and 2. (a) Clean. (b) Noisy. (c) BM4D Maggioni2013BM4D. (d) MTSNMF Ye2015MTSNMF. (e) LLRT Chang2017LLRT. (f) NGMeet He2022NGMeet. (g) LRMR Zhang2014LRMR. (h) E-3DTV Peng2020E-3DTV. (i) 3DlogTNN Zheng20203DlogTNN. (j) SST li2022spatialspectral. (k) TRQ3D Pang2022TRQ3DNet. (l) SERT li2023spectral (m) T3SC bodrito2021T3SC. (n) MTSNN++ xiong2023multitask. (o) DECSC.
• Figure 5: Row mean profiles of band 28 for the $Nachal\_0823-1214$ HSI from the ICVL dataset under the non-i.i.d. Gaussian noise with $\sigma \in [0,95]$. (a) Noisy. (b) BM4D Maggioni2013BM4D. (c) MTSNMF Ye2015MTSNMF. (d) LLRT Chang2017LLRT. (e) NGMeet He2022NGMeet. (f) LRMR Zhang2014LRMR. (g) E-3DTV Peng2020E-3DTV. (h) 3DlogTNN Zheng20203DlogTNN. (i) SST li2022spatialspectral. (j) TRQ3D Pang2022TRQ3DNet. (k) SERT li2023spectral. (l) T3SC bodrito2021T3SC. (m) MTSNN++ xiong2023multitask. (n) DECSC.