Neural Shrödinger Bridge Matching for Pansharpening

TL;DR

The paper tackles diffusion-based pansharpening by reformulating it as an inverse problem and addressing two core drawbacks of prior DPM approaches: (i) sampling from Gaussian noise neglects the LRMS prior, and (ii) inefficient sampling demands many steps. It introduces Simulation-free Schrödinger Bridge Matching, deriving SB SDEs/ODEs with linear diffusion components and using SB to align degradation and restoration between LRMS and HRMS under pan-sharp constraints. A specialized SBM-Net encoder-decoder and SB-specific block design enable efficient training and inference, achieving state-of-the-art results with as few as 1-step ODE or 5-step SDE sampling on standard pansharpening datasets. The approach also clarifies theoretical connections to OT, Flow Matching, and other SB methods, while providing practical benefits like analytic posterior forms and constraint-aware sampling. Overall, the SB framework yields faster, higher-fidelity pansharpening that leverages known degradation priors and distills SB/OT insights into an effective, scalable deep-learning solution.

Abstract

Recent diffusion probabilistic models (DPM) in the field of pansharpening have been gradually gaining attention and have achieved state-of-the-art (SOTA) performance. In this paper, we identify shortcomings in directly applying DPMs to the task of pansharpening as an inverse problem: 1) initiating sampling directly from Gaussian noise neglects the low-resolution multispectral image (LRMS) as a prior; 2) low sampling efficiency often necessitates a higher number of sampling steps. We first reformulate pansharpening into the stochastic differential equation (SDE) form of an inverse problem. Building upon this, we propose a Schrödinger bridge matching method that addresses both issues. We design an efficient deep neural network architecture tailored for the proposed SB matching. In comparison to the well-established DL-regressive-based framework and the recent DPM framework, our method demonstrates SOTA performance with fewer sampling steps. Moreover, we discuss the relationship between SB matching and other methods based on SDEs and ordinary differential equations (ODEs), as well as its connection with optimal transport. Code will be available.

Figures (10)

• Figure 1: Illustration of diffusion framework. It connects the Gaussian distribution with the HRMS distribution, which is inefficient in handling the pansharpening task. Previous works pandiffDDIF adopt this scheme.
• Figure 2: Main concept of the proposed SB SDE and ODE. HRMS is degraded by the forward SB SDE/ODE process (denoted as $q$ process) and recovered by the backward process (denoted as $p$ process). PAN, as the condition in the SB SDE or ODE, is omitted for a clear illustration.
• Figure 3: $\beta_t$, $\mu_t$, $\sigma_t$ and variance schedules of the proposed SB matching.
• Figure 4: Overall architecture of the proposed SBM-Net for SB SDE learning. The SBM-Net is an encoder-decoder architecture. Timestep $t$ is injected into every SBM blocks. C denotes tensor concatenation. Dash lines represent the U-Net-like shortcut connection. Valid lines are data flows.
• Figure 5: Illustration of the proposed efficient SBM block. SBM block shares a similar design scheme to the Metaformer block, composed of a token mixer and an FFN.

Theorems & Definitions (17)

• Definition 2.1: Stochastic degradation equation
• Proposition 2.2
• proof
• Remark 2.3
• Theorem 2.4
• proof
• Corollary 2.5
• Proposition 2.6: Analytic posterior when given bridge endpoints
• proof
• Corollary 2.7: Analytic posterior of SB ODE