Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic-CD: Remote Sensing Image Semantic Change Detection towards Open-vocabulary Setting

Yongshuo Zhu, Lu Li, Keyan Chen, Chenyang Liu, Fugen Zhou, Zhenwei Shi

TL;DR

This paper tackles semantic change detection in remote sensing by addressing the limited generalization of traditional binary change detection methods. It proposes Semantic-CD, a CLIP-guided framework that decouples binary and semantic change detection, employing a bi-temporal CLIP encoder, an open semantic prompter to create semantic cost volumes, and separate decoders for binary and semantic outputs. Key contributions include integrating a change semantic adapter into CLIP (BCSF), an instance-level open-vocabulary prompter, and a fully decoupled two-stage training scheme, validated on the SECOND dataset. Results demonstrate improved semantic mask quality and reduced semantic classification errors, highlighting the practical value of applying vision-language foundation model priors to open-vocabulary SCD in remote sensing.

Abstract

Remote sensing image semantic change detection is a method used to analyze remote sensing images, aiming to identify areas of change as well as categorize these changes within images of the same location taken at different times. Traditional change detection methods often face challenges in generalizing across semantic categories in practical scenarios. To address this issue, we introduce a novel approach called Semantic-CD, specifically designed for semantic change detection in remote sensing images. This method incorporates the open vocabulary semantics from the vision-language foundation model, CLIP. By utilizing CLIP's extensive vocabulary knowledge, our model enhances its ability to generalize across categories and improves segmentation through fully decoupled multi-task learning, which includes both binary change detection and semantic change detection tasks. Semantic-CD consists of four main components: a bi-temporal CLIP visual encoder for extracting features from bi-temporal images, an open semantic prompter for creating semantic cost volume maps with open vocabulary, a binary change detection decoder for generating binary change detection masks, and a semantic change detection decoder for producing semantic labels. Experimental results on the SECOND dataset demonstrate that Semantic-CD achieves more accurate masks and reduces semantic classification errors, illustrating its effectiveness in applying semantic priors from vision-language foundation models to SCD tasks.

Semantic-CD: Remote Sensing Image Semantic Change Detection towards Open-vocabulary Setting

TL;DR

This paper tackles semantic change detection in remote sensing by addressing the limited generalization of traditional binary change detection methods. It proposes Semantic-CD, a CLIP-guided framework that decouples binary and semantic change detection, employing a bi-temporal CLIP encoder, an open semantic prompter to create semantic cost volumes, and separate decoders for binary and semantic outputs. Key contributions include integrating a change semantic adapter into CLIP (BCSF), an instance-level open-vocabulary prompter, and a fully decoupled two-stage training scheme, validated on the SECOND dataset. Results demonstrate improved semantic mask quality and reduced semantic classification errors, highlighting the practical value of applying vision-language foundation model priors to open-vocabulary SCD in remote sensing.

Abstract

Remote sensing image semantic change detection is a method used to analyze remote sensing images, aiming to identify areas of change as well as categorize these changes within images of the same location taken at different times. Traditional change detection methods often face challenges in generalizing across semantic categories in practical scenarios. To address this issue, we introduce a novel approach called Semantic-CD, specifically designed for semantic change detection in remote sensing images. This method incorporates the open vocabulary semantics from the vision-language foundation model, CLIP. By utilizing CLIP's extensive vocabulary knowledge, our model enhances its ability to generalize across categories and improves segmentation through fully decoupled multi-task learning, which includes both binary change detection and semantic change detection tasks. Semantic-CD consists of four main components: a bi-temporal CLIP visual encoder for extracting features from bi-temporal images, an open semantic prompter for creating semantic cost volume maps with open vocabulary, a binary change detection decoder for generating binary change detection masks, and a semantic change detection decoder for producing semantic labels. Experimental results on the SECOND dataset demonstrate that Semantic-CD achieves more accurate masks and reduces semantic classification errors, illustrating its effectiveness in applying semantic priors from vision-language foundation models to SCD tasks.
Paper Structure (15 sections, 6 equations, 3 figures, 1 table)

This paper contains 15 sections, 6 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Overall Architecture
  4. Bi-temporal CLIP visual encoder,
  5. Open Semantic Prompter
  6. BCD Decoder
  7. SCD Decoder
  8. Training Strategy
  9. Experimental Results and Analyses
  10. Experimental Setup
  11. Dataset and Evaluation Metrics
  12. Implementation Details
  13. Experimental Results
  14. Discussion
  15. Conclusion

Figures (3)

  • Figure 1: The architecture of the Semantic-CD consists of four main components: an adapted CLIP vision encoder for extracting robust features from bi-temporal images; a Prompt Learner tasked with generating visual embeddings that encapsulate textual semantics; a BCD decoder responsible for producing binary detection masks; and a SCD decoder which assigns precise semantic labels to these binary masks.
  • Figure 2: The structure of the Semantic Change Detection Decoder.
  • Figure 3: Qualitative comparison results with other methods on the SECOND dataset.