Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-Domain Separable Translation Network for Multimodal Image Change Detection

Tao Zhan, Yuanyuan Zhu, Jie Lan, Qianlong Dang

TL;DR

A novel unsupervised cross-domain separable translation network (CSTN) is proposed, which uniquely integrates a within-domain self-reconstruction and a cross-domain image translation and cycle-reconstruction workflow with change detection constraints.

Abstract

In the remote sensing community, multimodal change detection (MCD) is particularly critical due to its ability to track changes across different imaging conditions and sensor types, making it highly applicable to a wide range of real-world scenarios. This paper focuses on addressing the challenges of MCD, especially the difficulty in comparing images from different sensors with varying styles and statistical characteristics of geospatial objects. Traditional MCD methods often struggle with these variations, leading to inaccurate and unreliable results. To overcome these limitations, a novel unsupervised cross-domain separable translation network (CSTN) is proposed, which uniquely integrates a within-domain self-reconstruction and a cross-domain image translation and cycle-reconstruction workflow with change detection constraints. The model is optimized by implementing both the tasks of image translation and MCD simultaneously, thereby guaranteeing the comparability of learned features from multimodal images. Specifically, a simple yet efficient dual-branch convolutional architecture is employed to separate the content and style information of multimodal images. This process generates a style-independent content-comparable feature space, which is crucial for achieving accurate change detection even in the presence of significant sensor variations. Extensive experimental results demonstrate the effectiveness of the proposed method, showing remarkable improvements over state-of-the-art approaches in terms of accuracy and efficacy for MCD. The implementation of our method will be publicly available at \url{https://github.com/OMEGA-RS/CSTN}

Cross-Domain Separable Translation Network for Multimodal Image Change Detection

TL;DR

A novel unsupervised cross-domain separable translation network (CSTN) is proposed, which uniquely integrates a within-domain self-reconstruction and a cross-domain image translation and cycle-reconstruction workflow with change detection constraints.

Abstract

In the remote sensing community, multimodal change detection (MCD) is particularly critical due to its ability to track changes across different imaging conditions and sensor types, making it highly applicable to a wide range of real-world scenarios. This paper focuses on addressing the challenges of MCD, especially the difficulty in comparing images from different sensors with varying styles and statistical characteristics of geospatial objects. Traditional MCD methods often struggle with these variations, leading to inaccurate and unreliable results. To overcome these limitations, a novel unsupervised cross-domain separable translation network (CSTN) is proposed, which uniquely integrates a within-domain self-reconstruction and a cross-domain image translation and cycle-reconstruction workflow with change detection constraints. The model is optimized by implementing both the tasks of image translation and MCD simultaneously, thereby guaranteeing the comparability of learned features from multimodal images. Specifically, a simple yet efficient dual-branch convolutional architecture is employed to separate the content and style information of multimodal images. This process generates a style-independent content-comparable feature space, which is crucial for achieving accurate change detection even in the presence of significant sensor variations. Extensive experimental results demonstrate the effectiveness of the proposed method, showing remarkable improvements over state-of-the-art approaches in terms of accuracy and efficacy for MCD. The implementation of our method will be publicly available at \url{https://github.com/OMEGA-RS/CSTN}
Paper Structure (40 sections, 18 equations, 14 figures, 10 tables, 1 algorithm)

This paper contains 40 sections, 18 equations, 14 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Related Work
  4. Classification-Based Methods
  5. Handcrafted Feature-Based Methods
  6. Deep Latent Feature-Based Methods
  7. Image Translation-Based Methods
  8. Motivation
  9. Contribution
  10. Outline
  11. Methodology
  12. Overview
  13. Model Architecture
  14. Content Encoder
  15. Style Encoder
  16. ...and 25 more sections

Figures (14)

  • Figure 1: Overview of the proposed CSTN framework. During the training stage, the model is optimized by jointly executing following two workflows: (a) within-domain image self-reconstruction and (b) cross-domain image translation and cycle-reconstruction, aiming to acquire comparable content features from multimodal images by establishing a style-independent feature space. In the inference stage, (c) the change detection pipeline is performed to derive a binary change map by making full use of the content features from the multimodal mages and translated images.
  • Figure 2: Scheme of cross-domain image translation (taking domain $\undefined{X}$→ domain $\undefined{Y}$ as an example).
  • Figure 3: Structure of FFB.
  • Figure 4: Sardinia dataset. (a) Near-infrared image acquired in 1995. (b) Optical image acquired in 1996. (c) Ground truth.
  • Figure 5: Texas dataset. (a) Optical image acquired in 2011. (b) Optical image acquired in 2011. (c) Ground truth. False color composites are shown for both the images.
  • ...and 9 more figures