Diffusion Posterior Sampler for Hyperspectral Unmixing with Spectral Variability Modeling

TL;DR

This work tackles hyperspectral unmixing under spectral variability by introducing DPS4Un, a conditional diffusion posterior sampler that learns endmember priors from region-specific, superpixel-based spectral bundles. By decoupling prior learning from posterior sampling and conditioning on per-superpixel IDs, the method achieves region-aware spectral variability modeling and data fidelity, enabling refined abundance and endmember estimation. The approach demonstrates superior performance on three real datasets and provides insightful sampling trajectories that visualize the diffusion toward high-probability endmember distributions. The combination of a diffusion-based prior with region-wise optimization offers a principled Bayesian framework for robust, variability-aware SU in complex scenes.

Abstract

Linear spectral mixture models (LMM) provide a concise form to disentangle the constituent materials (endmembers) and their corresponding proportions (abundance) in a single pixel. The critical challenges are how to model the spectral prior distribution and spectral variability. Prior knowledge and spectral variability can be rigorously modeled under the Bayesian framework, where posterior estimation of Abundance is derived by combining observed data with endmember prior distribution. Considering the key challenges and the advantages of the Bayesian framework, a novel method using a diffusion posterior sampler for semiblind unmixing, denoted as DPS4Un, is proposed to deal with these challenges with the following features: (1) we view the pretrained conditional spectrum diffusion model as a posterior sampler, which can combine the learned endmember prior with observation to get the refined abundance distribution. (2) Instead of using the existing spectral library as prior, which may raise bias, we establish the image-based endmember bundles within superpixels, which are used to train the endmember prior learner with diffusion model. Superpixels make sure the sub-scene is more homogeneous. (3) Instead of using the image-level data consistency constraint, the superpixel-based data fidelity term is proposed. (4) The endmember is initialized as Gaussian noise for each superpixel region, DPS4Un iteratively updates the abundance and endmember, contributing to spectral variability modeling. The experimental results on three real-world benchmark datasets demonstrate that DPS4Un outperforms the state-of-the-art hyperspectral unmixing methods.

Figures (14)

• Figure 1: Sampling trajectory of our proposed DPS4Un from Gaussian noise (left) to endmember distribution (right) considering the spectral variability (multiple trajectories). The $x$-axis represents the DDIM sampling timestep $t$, total 20 steps, the $y$-axis represents the value of the denoised endmember $\boldsymbol{A}_t$. The color is the probability density of that value at that timestep. Overall, the reverse process should flow toward high probability regions of the data distribution, showing that our proposed DPS4Un can path directly to the high-mass target region, which is good for the spectral unmixing problem. To show the high-dimensional data, PCA is applied to reduce the data into a 2D space.
• Figure 2: Illustration for region-based spectral library construction. (a) HSI in HSV color space. (b) superpixel (c) superpixel overlay on HSI. (d) Data distribution of mixed pixels (grey) and spectral library (colored) in 2D PCA space.
• Figure 3: Architecture overview of the MLP-based denoising network $\boldsymbol{s}_{\theta}(\boldsymbol{A}_t, t, c)$. $\Phi$ is the Sinusodial Timestep Embedding and Embed(c) represents the label embedding layer.
• Figure 4: Sampling process (from left to right) of $\nabla_{\boldsymbol{A}_t} \log p(\hat{\boldsymbol{S}}_0|\hat{\boldsymbol{A}_0})$ (abundance updating, from FCLSU to maximum posterior using DPS4Un), and $\boldsymbol{s}_{\theta^*} (\boldsymbol{A}_t, t, c)$ (endmember updating, from Gaussian noise to data distribution). The abundance map is initialized from the FCLUS algorithm; basically, it is random. With our pretrained diffusion model under the cluster ID condition, it becomes more fine-grained. Take Jasper Ridge dataset as an example. Each column from time step 951, 701, 451, 201, 1 using DDIM.
• Figure 5: Estimated abundances on the Jasper Ridge dataset.