Enpowering Your Pansharpening Models with Generalizability: Unified Distribution is All You Need

TL;DR

UniPAN tackles the generalization gap in deep pansharpening caused by cross-sensor distribution shifts by introducing a model-agnostic distribution transformation that maps data from multiple sensors to a unified target distribution using inverse transform sampling. Grounded in optimal transport theory, the method operates in 1D per spectral band via quantile-based transformations and requires no additional trainable parameters, enabling train-once, deploy-forever performance across diverse satellites. Extensive experiments across six satellites and twelve models demonstrate consistent cross-sensor improvements, with the uniform target distribution $\mathcal{U}(0,1)$ often yielding the strongest gains, albeit with minor spectral distortions that are outweighed by spatial fidelity benefits. The work highlights a practical, scalable pathway to robust remote-sensing fusion by aligning distributions in data space, with potential extensions to multidimensional settings and other fusion tasks.

Abstract

Existing deep learning-based models for remote sensing pansharpening exhibit exceptional performance on training datasets. However, due to sensor-specific characteristics and varying imaging conditions, these models suffer from substantial performance degradation when applied to unseen satellite data, lacking generalizability and thus limiting their applicability. We argue that the performance drops stem primarily from distributional discrepancies from different sources and the key to addressing this challenge lies in bridging the gap between training and testing distributions. To validate the idea and further achieve a "train once, deploy forever" capability, this paper introduces a novel and intuitive approach to enpower any pansharpening models with generalizability by employing a unified distribution strategy (UniPAN). Specifically, we construct a distribution transformation function that normalizes the pixels sampled from different sources to conform to an identical distribution. The deep models are trained on the transformed domain, and during testing on new datasets, the new data are also transformed to match the training distribution. UniPAN aims to train and test the model on a unified and consistent distribution, thereby enhancing its generalizability. Extensive experiments validate the efficacy of UniPAN, demonstrating its potential to significantly enhance the performance of deep pansharpening models across diverse satellite sensors. Codes: https://github.com/yc-cui/UniPAN.

Figures (7)

• Figure 1: Comparison of generalization metrics w/ and w/o UniPAN trained on WorldView-3 (WV3) using FusionNet FusionNet.
• Figure 2: Comparison of tSNE visualization w/ and w/o UniPAN.
• Figure 3: Generalization testing on GaoFen-1, the networks are trained on WorldView-3. The bright green box indicates the zoomed-in area. Below the zoomed-in image, the first line is the QNR value, and the second line is the $D_{\lambda}$/$D_S$ value. In most cases, the use of UniPAN yields superior spectral and spatial preservation. For instance, MDCUN MDCUN exhibits a highly close alignment with the LRMS in spectral characteristics with UniPAN of $\rho_t = \mathcal{N}(0,1)$.
• Figure 4: Full-resolution generalization testing on QuickBird. The bright green box indicates the zoomed-in area. Below the zoomed-in image, the first line is the QNR value, and the second line is the $D_{\lambda}$/$D_S$ value.
• Figure 5: Comparison of data distributions before and after applying UniPAN. The upper-right and diagonals of each subplot display the kernel density estimation (KDE) curves of Green band for each satellite. The scatter plots in the lower-left areas of each subplots represent quantile-quantile (Q-Q) plots of green band, with the dashed line indicating $y=x$. The greater the deviation from the $y=x$ line, the more significant the disparity between the two distributions. Zoom in for better view.