Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeurOp-Diff:Continuous Remote Sensing Image Super-Resolution via Neural Operator Diffusion

Zihao Xu, Yuzhi Tang, Bowen Xu, Qingquan Li

TL;DR

This work tackles the limitation of fixed-scale remote sensing super-resolution by introducing NeurOp-Diff, a diffusion model guided by neural operators to achieve continuous, high-fidelity SR. The method encodes low-resolution inputs through a neural operator to provide a rich, scale-agnostic prior, which conditions the reverse diffusion steps and mitigates artifacts common in prior SR approaches. Key contributions include the integration of Galerkin-type neural operators with conditional diffusion, a scale-adaptive training regime (up to $M=8$) and extensive evaluation on UCMerced, AID, and RSSCN7, showing improvements in PSNR, SSIM, and perceptual quality while enabling continuous magnification beyond training scales. The approach offers practical benefits for multi-scale data fusion and flexible magnification in remote sensing, with code available at the provided repository.

Abstract

Most publicly accessible remote sensing data suffer from low resolution, limiting their practical applications. To address this, we propose a diffusion model guided by neural operators for continuous remote sensing image super-resolution (NeurOp-Diff). Neural operators are used to learn resolution representations at arbitrary scales, encoding low-resolution (LR) images into high-dimensional features, which are then used as prior conditions to guide the diffusion model for denoising. This effectively addresses the artifacts and excessive smoothing issues present in existing super-resolution (SR) methods, enabling the generation of high-quality, continuous super-resolution images. Specifically, we adjust the super-resolution scale by a scaling factor s, allowing the model to adapt to different super-resolution magnifications. Furthermore, experiments on multiple datasets demonstrate the effectiveness of NeurOp-Diff. Our code is available at https://github.com/zerono000/NeurOp-Diff.

NeurOp-Diff:Continuous Remote Sensing Image Super-Resolution via Neural Operator Diffusion

TL;DR

This work tackles the limitation of fixed-scale remote sensing super-resolution by introducing NeurOp-Diff, a diffusion model guided by neural operators to achieve continuous, high-fidelity SR. The method encodes low-resolution inputs through a neural operator to provide a rich, scale-agnostic prior, which conditions the reverse diffusion steps and mitigates artifacts common in prior SR approaches. Key contributions include the integration of Galerkin-type neural operators with conditional diffusion, a scale-adaptive training regime (up to ) and extensive evaluation on UCMerced, AID, and RSSCN7, showing improvements in PSNR, SSIM, and perceptual quality while enabling continuous magnification beyond training scales. The approach offers practical benefits for multi-scale data fusion and flexible magnification in remote sensing, with code available at the provided repository.

Abstract

Most publicly accessible remote sensing data suffer from low resolution, limiting their practical applications. To address this, we propose a diffusion model guided by neural operators for continuous remote sensing image super-resolution (NeurOp-Diff). Neural operators are used to learn resolution representations at arbitrary scales, encoding low-resolution (LR) images into high-dimensional features, which are then used as prior conditions to guide the diffusion model for denoising. This effectively addresses the artifacts and excessive smoothing issues present in existing super-resolution (SR) methods, enabling the generation of high-quality, continuous super-resolution images. Specifically, we adjust the super-resolution scale by a scaling factor s, allowing the model to adapt to different super-resolution magnifications. Furthermore, experiments on multiple datasets demonstrate the effectiveness of NeurOp-Diff. Our code is available at https://github.com/zerono000/NeurOp-Diff.
Paper Structure (20 sections, 14 equations, 5 figures, 5 tables)

This paper contains 20 sections, 14 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Relate Work
  3. Method
  4. Conditional Denoising Diffusion Model
  5. Forward Process
  6. Reverse Process
  7. Neural Operator
  8. Neural Operator with Conditional Diffusion
  9. Network details
  10. Experiments
  11. Training
  12. Experimental Setup.
  13. Training Details.
  14. Comparison of Generative SR
  15. Qualitative Comparison
  16. ...and 5 more sections

Figures (5)

  • Figure 1: The figure illustrates the latent representations learned by the diffusion model after processing the low-resolution (LR) image using different methods, including Bicubic interpolation, EDSR, and Neural Operator (NO).
  • Figure 2: (a) The architecture of NeurOp-Diff. The noisy image is denoised through T iterations to generate a clear image. $s$ is the scaling factor. (b) Neural operator architecture. The LR image is first encoded into high-dimensional features through the encoder (E), interpolation, and lifting functions ($\mathcal{L}$), which linearly transform low-resolution pixel values. Then, it passes through a kernel integral composed of Galerkin-type attention to produce the output features. Finally, MPL is applied for channel transformation. (c) The integration of neural operators and diffusion models. Here, $Q$, $K$, and $V$ represent the components of the attention mechanism.
  • Figure 3: A visualization quality comparison between NeurOp-Diff and generative super-resolution (SR) models on UCMerced and AID. For each image, we also display finer details of the magnified regions.
  • Figure 4: Qualitative comparison on 4× SR on UCMerced. Compared to regression-based methods, NeurOp-Diff is able to reconstruct more details and produce sharper textures.
  • Figure 5: Visualization of continuous SR results on UCMerced. The resolution of the ground truth image is $256 \times 256$. We randomly selected multiple magnification factors within the range of (1, 8] for demonstration, additionally including two out-of-distribution magnification factors. (i.e., 5.8x, 8x, and 10x).