Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Spectral Diffusion Prior for Hyperspectral Image Super-Resolution

Jianjun Liu, Zebin Wu, Liang Xiao

TL;DR

This work designs a spectral diffusion model (SDM) to capture the spectral distribution of HSIs and exploit it as a prior for the problem of unsupervised HSI super-resolution and transfers the spectral distribution knowledge of the trained SDM by means of keeping its transition information to induce a regularization term in the framework of maximum a posteriori.

Abstract

Fusion-based hyperspectral image (HSI) super-resolution aims to produce a high-spatial-resolution HSI by fusing a low-spatial-resolution HSI and a high-spatial-resolution multispectral image. Such a HSI super-resolution process can be modeled as an inverse problem, where the prior knowledge is essential for obtaining the desired solution. Motivated by the success of diffusion models, we propose a novel spectral diffusion prior for fusion-based HSI super-resolution. Specifically, we first investigate the spectrum generation problem and design a spectral diffusion model to model the spectral data distribution. Then, in the framework of maximum a posteriori, we keep the transition information between every two neighboring states during the reverse generative process, and thereby embed the knowledge of trained spectral diffusion model into the fusion problem in the form of a regularization term. At last, we treat each generation step of the final optimization problem as its subproblem, and employ the Adam to solve these subproblems in a reverse sequence. Experimental results conducted on both synthetic and real datasets demonstrate the effectiveness of the proposed approach. The code of the proposed approach will be available on https://github.com/liuofficial/SDP.

A Spectral Diffusion Prior for Hyperspectral Image Super-Resolution

TL;DR

This work designs a spectral diffusion model (SDM) to capture the spectral distribution of HSIs and exploit it as a prior for the problem of unsupervised HSI super-resolution and transfers the spectral distribution knowledge of the trained SDM by means of keeping its transition information to induce a regularization term in the framework of maximum a posteriori.

Abstract

Fusion-based hyperspectral image (HSI) super-resolution aims to produce a high-spatial-resolution HSI by fusing a low-spatial-resolution HSI and a high-spatial-resolution multispectral image. Such a HSI super-resolution process can be modeled as an inverse problem, where the prior knowledge is essential for obtaining the desired solution. Motivated by the success of diffusion models, we propose a novel spectral diffusion prior for fusion-based HSI super-resolution. Specifically, we first investigate the spectrum generation problem and design a spectral diffusion model to model the spectral data distribution. Then, in the framework of maximum a posteriori, we keep the transition information between every two neighboring states during the reverse generative process, and thereby embed the knowledge of trained spectral diffusion model into the fusion problem in the form of a regularization term. At last, we treat each generation step of the final optimization problem as its subproblem, and employ the Adam to solve these subproblems in a reverse sequence. Experimental results conducted on both synthetic and real datasets demonstrate the effectiveness of the proposed approach. The code of the proposed approach will be available on https://github.com/liuofficial/SDP.
Paper Structure (17 sections, 16 equations, 11 figures, 6 tables, 1 algorithm)

This paper contains 17 sections, 16 equations, 11 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. HSI super-resolution
  4. Diffusion Models
  5. Proposed Approach
  6. Spectral Diffusion Model
  7. HSI Super-Resolution with Spectral Diffusion Prior
  8. Optimization
  9. Experimental Results and Analysis
  10. Synthetic Datasets
  11. Implementation Details and Influence of Parameters
  12. Experiment Results on Synthetic Datasets
  13. Relationship between FID and Fusion Accuracy
  14. Comparison With the State of the Art
  15. Computational Efficiency
  16. ...and 2 more sections

Figures (11)

  • Figure 1: Illustration of spectral diffusion model.
  • Figure 2: Architecture overview of MLP-based denoising network.
  • Figure 3: FIDs as a function of timestep $t$ when applied to the given three datasets.
  • Figure 4: PSNR with respect to parameters $\lambda$ and $\gamma$. (a) PaviaU dataset. (b) KSC dataset. (c) DC dataset.
  • Figure 5: Illustration the relationship between FID and PSNR. (a) PaviaU dataset. (b) KSC dataset. (c) DC dataset.
  • ...and 6 more figures