Comprehensive Analysis and Improvements in Pansharpening Using Deep Learning

TL;DR

The paper tackles spectral distortions in pansharpening by enhancing PSGAN with multiple regularizers that target spectral fidelity. It introduces SAM-based regularization, multi-resolution SAM, perceptual losses based on high-level features and Gram matrices, and a Gram-based reconstruction loss, integrating them into a final objective $L=\eta_1 L(G)+\eta_2 Regularizer$. Across WorldView-3 data, Gram-matrix reconstruction loss delivers broad performance gains across metrics, while SAM-based loss yields the strongest spectral improvements; perceptual losses help particularly on smaller datasets. The work provides practical guidance for improving DL-based pansharpening, achieving better spectral preservation and visually convincing results with notable gains in quantitative performance.

Abstract

Pansharpening is a crucial task in remote sensing, enabling the generation of high-resolution multispectral images by fusing low-resolution multispectral data with high-resolution panchromatic images. This paper provides a comprehensive analysis of traditional and deep learning-based pansharpening methods. While state-of-the-art deep learning methods have significantly improved image quality, issues like spectral distortions persist. To address this, we propose enhancements to the PSGAN framework by introducing novel regularization techniques for the generator loss function. Experimental results on images from the Worldview-3 dataset demonstrate that the proposed modifications improve spectral fidelity and achieve superior performance across multiple quantitative metrics while delivering visually superior results.

Figures (5)

• Figure 1: PanNet model
• Figure 2: PSGAN Generator network
• Figure 3: PSGAN discriminator network
• Figure 4: Network to generate high level features of multispectral images
• Figure 5: Pansharpening results on Test dataset