Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-resolution Multi-spectral Image Guided DEM Super-resolution using Sinkhorn Regularized Adversarial Network

Subhajit Paul, Ashutosh Gupta

TL;DR

This study intends to address the generation of high-resolution DEMs using high- resolution multi-spectral (MX) satellite imagery by incorporating adversarial learning by utilizing the notion of polarized self-attention of discriminator spatial maps as well as introducing a Densely connected Multi-Residual Block (DMRB) module to assist in efficient gradient flow.

Abstract

Digital Elevation Model (DEM) is an essential aspect in the remote sensing domain to analyze and explore different applications related to surface elevation information. In this study, we intend to address the generation of high-resolution DEMs using high-resolution multi-spectral (MX) satellite imagery by incorporating adversarial learning. To promptly regulate this process, we utilize the notion of polarized self-attention of discriminator spatial maps as well as introduce a Densely connected Multi-Residual Block (DMRB) module to assist in efficient gradient flow. Further, we present an objective function related to optimizing Sinkhorn distance with traditional GAN to improve the stability of adversarial learning. In this regard, we provide both theoretical and empirical substantiation of better performance in terms of vanishing gradient issues and numerical convergence. We demonstrate both qualitative and quantitative outcomes with available state-of-the-art methods. Based on our experiments on DEM datasets of Shuttle Radar Topographic Mission (SRTM) and Cartosat-1, we show that the proposed model performs preferably against other learning-based state-of-the-art methods. We also generate and visualize several high-resolution DEMs covering terrains with diverse signatures to show the performance of our model.

High-resolution Multi-spectral Image Guided DEM Super-resolution using Sinkhorn Regularized Adversarial Network

TL;DR

This study intends to address the generation of high-resolution DEMs using high- resolution multi-spectral (MX) satellite imagery by incorporating adversarial learning by utilizing the notion of polarized self-attention of discriminator spatial maps as well as introducing a Densely connected Multi-Residual Block (DMRB) module to assist in efficient gradient flow.

Abstract

Digital Elevation Model (DEM) is an essential aspect in the remote sensing domain to analyze and explore different applications related to surface elevation information. In this study, we intend to address the generation of high-resolution DEMs using high-resolution multi-spectral (MX) satellite imagery by incorporating adversarial learning. To promptly regulate this process, we utilize the notion of polarized self-attention of discriminator spatial maps as well as introduce a Densely connected Multi-Residual Block (DMRB) module to assist in efficient gradient flow. Further, we present an objective function related to optimizing Sinkhorn distance with traditional GAN to improve the stability of adversarial learning. In this regard, we provide both theoretical and empirical substantiation of better performance in terms of vanishing gradient issues and numerical convergence. We demonstrate both qualitative and quantitative outcomes with available state-of-the-art methods. Based on our experiments on DEM datasets of Shuttle Radar Topographic Mission (SRTM) and Cartosat-1, we show that the proposed model performs preferably against other learning-based state-of-the-art methods. We also generate and visualize several high-resolution DEMs covering terrains with diverse signatures to show the performance of our model.
Paper Structure (19 sections, 5 theorems, 41 equations, 14 figures, 3 tables)

This paper contains 19 sections, 5 theorems, 41 equations, 14 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Network Architecture
  4. Formulation of objective function
  5. Experiments and results analysis
  6. Dataset and implementation details
  7. Results analysis
  8. Conclusion
  9. Acknowledgement
  10. Appendix: Smoothness of Sinkhorn loss
  11. Appendix: Upper-bound of expected gradient in SIRAN set-up
  12. Appendix: Iteration complexiety of SIRAN
  13. Simplified Theorem 3
  14. Appendix: Details of used losses
  15. Appendix: Implementation Details
  16. ...and 4 more sections

Key Result

Theorem 1

Consider the Sinkhorn loss $\mathcal{S}_{C,\varepsilon}(\mu_\theta, \nu)$ between two measures $\mu_\theta$ and $\nu$ on $\mathcal{X}$ and $\mathcal{Y}$ two bounded subsets of $\mathbb{R}^{d}$, with a $\mathcal{C}^{\infty}$, $L_0$-Lipschitz, and $L_1$-smooth cost function $C$. Then, for $(\theta_1, where $L$ is the Lipschitz in $\theta$ corresponding to $\mathbf{G}$, $\kappa = 2(L_0 |\mathcal{X}|

Figures (14)

  • Figure 1: Overview of the proposed adversarial framework. (a) The generator $\mathbf{G}$ takes discriminative spatial attention from (b) discriminator $\mathbf{D}$ as conditional input, paased via a (d) Polarized Self-Attention (PSA) block. Both $\mathbf{G}$ and $\mathbf{D}$ constitute of (c) Densely connected Multi Residual Blocks (DMRBs) with residual convolution block (RCB) as the building block.
  • Figure 2: Results for DEM super-resolution (better viewed at 200%) for both inside and ouside India data and comparisons with other baseline methods.
  • Figure 3: Line profile analysis of SIRAN and other baselines.
  • Figure 4: 3-D visualization of Super-resolved and SRTM DEM (better at 200%)
  • Figure 5: (a) Source, (b) Target, (c)-(h) Discriminator spatial attention after each DMRB from top to bottom.
  • ...and 9 more figures

Theorems & Definitions (9)

  • Theorem 1: Smoothness of Sinkhorn loss
  • proof
  • Lemma A.1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Corollary 1