Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model

Runmin Dong, Shuai Yuan, Bin Luo, Mengxuan Chen, Jinxiao Zhang, Lixian Zhang, Weijia Li, Juepeng Zheng, Haohuan Fu

TL;DR

A change-aware diffusion model named Ref-Diff for RefSR is proposed, using the land cover change priors to guide the denoising process explicitly, and injects the priors into the denoising model to improve the utilization of reference information in unchanged areas and regulate the reconstruction of semantically relevant content in changed areas.

Abstract

Reference-based super-resolution (RefSR) has the potential to build bridges across spatial and temporal resolutions of remote sensing images. However, existing RefSR methods are limited by the faithfulness of content reconstruction and the effectiveness of texture transfer in large scaling factors. Conditional diffusion models have opened up new opportunities for generating realistic high-resolution images, but effectively utilizing reference images within these models remains an area for further exploration. Furthermore, content fidelity is difficult to guarantee in areas without relevant reference information. To solve these issues, we propose a change-aware diffusion model named Ref-Diff for RefSR, using the land cover change priors to guide the denoising process explicitly. Specifically, we inject the priors into the denoising model to improve the utilization of reference information in unchanged areas and regulate the reconstruction of semantically relevant content in changed areas. With this powerful guidance, we decouple the semantics-guided denoising and reference texture-guided denoising processes to improve the model performance. Extensive experiments demonstrate the superior effectiveness and robustness of the proposed method compared with state-of-the-art RefSR methods in both quantitative and qualitative evaluations. The code and data are available at https://github.com/dongrunmin/RefDiff.

Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model

TL;DR

A change-aware diffusion model named Ref-Diff for RefSR is proposed, using the land cover change priors to guide the denoising process explicitly, and injects the priors into the denoising model to improve the utilization of reference information in unchanged areas and regulate the reconstruction of semantically relevant content in changed areas.

Abstract

Reference-based super-resolution (RefSR) has the potential to build bridges across spatial and temporal resolutions of remote sensing images. However, existing RefSR methods are limited by the faithfulness of content reconstruction and the effectiveness of texture transfer in large scaling factors. Conditional diffusion models have opened up new opportunities for generating realistic high-resolution images, but effectively utilizing reference images within these models remains an area for further exploration. Furthermore, content fidelity is difficult to guarantee in areas without relevant reference information. To solve these issues, we propose a change-aware diffusion model named Ref-Diff for RefSR, using the land cover change priors to guide the denoising process explicitly. Specifically, we inject the priors into the denoising model to improve the utilization of reference information in unchanged areas and regulate the reconstruction of semantically relevant content in changed areas. With this powerful guidance, we decouple the semantics-guided denoising and reference texture-guided denoising processes to improve the model performance. Extensive experiments demonstrate the superior effectiveness and robustness of the proposed method compared with state-of-the-art RefSR methods in both quantitative and qualitative evaluations. The code and data are available at https://github.com/dongrunmin/RefDiff.
Paper Structure (19 sections, 3 equations, 5 figures, 4 tables)

This paper contains 19 sections, 3 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Reference-Based Super-Resolution Methods
  4. Conditional Diffusion Model for Super-Resolution
  5. Change Detection
  6. Methodology
  7. Preliminary
  8. Change-Aware Denoising Model
  9. Degradation Model and Implementation Details
  10. Experiments
  11. Datasets and Evaluation
  12. Comparison Results
  13. Ablation Study
  14. Experiments Using Land Cover Change Predictions
  15. Discussion of the Interaction between RefSR and CD Tasks
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Illustration of the proposed change-aware diffusion model for RefSR. LR is a low-resolution image and HR is the corresponding high-resolution image. Ref represents a geographically matched reference high-resolution image acquired at another time.
  • Figure 2: The architecture of the proposed change-aware denoising model. It consists of change-aware encoder and decoder blocks. The LR, Ref, and land cover change mask are combined with the noise input and are also injected into change-aware decoder blocks.
  • Figure 3: Comparison results on SECOND (a-c) and CNAM-CD (d-f) datasets with $8\times$ and $16\times$ scaling factors.
  • Figure 4: The results for two examples with mislabeled land cover change masks on CNAM-CD 8$\times$ and 16$\times$ datasets. (a) shows an example with false negative detection, and (b) shows an example with false positive detection.
  • Figure B5: Comparison results on two real datasets. (a-b) are located in Jiaxing, China. (c-d) are located in Rennes and Caen, France. (a) and (c) are with $8\times$ scaling factor. (b) and (d) are with $16\times$ scaling factor.