Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DynamicEarth: How Far are We from Open-Vocabulary Change Detection?

Kaiyu Li, Xiangyong Cao, Yupeng Deng, Chao Pang, Zepeng Xin, Deyu Meng, Zhi Wang

TL;DR

This work defines Open-Vocabulary Change Detection (OVCD) to localize and interpret changes between bi-temporal remote-sensing images for arbitrary categories described by text, addressing open-world requires no category-specific training. It introduces two training-free frameworks, M-C-I and I-M-C, that re-use off-the-shelf foundation models (e.g., SAM, DINO, CLIP, Grounding DINO) to detect, compare, and classify changes, avoiding extensive annotated data. Through experiments on multiple building and land-cover datasets, the authors show strong cross-dataset generalization and competitive performance against unsupervised and some supervised baselines, highlighting OVCD as a practical path toward open-world perception in Earth observation. The DynamicEarth codebase is released to enable researchers to reproduce results and extend OVCD methods in real-world scenarios.

Abstract

Monitoring Earth's evolving land covers requires methods capable of detecting changes across a wide range of categories and contexts. Existing change detection methods are hindered by their dependency on predefined classes, reducing their effectiveness in open-world applications. To address this issue, we introduce open-vocabulary change detection (OVCD), a novel task that bridges vision and language to detect changes across any category. Considering the lack of high-quality data and annotation, we propose two training-free frameworks, M-C-I and I-M-C, which leverage and integrate off-the-shelf foundation models for the OVCD task. The insight behind the M-C-I framework is to discover all potential changes and then classify these changes, while the insight of I-M-C framework is to identify all targets of interest and then determine whether their states have changed. Based on these two frameworks, we instantiate to obtain several methods, e.g., SAM-DINOv2-SegEarth-OV, Grounding-DINO-SAM2-DINO, etc. Extensive evaluations on 5 benchmark datasets demonstrate the superior generalization and robustness of our OVCD methods over existing supervised and unsupervised methods. To support continued exploration, we release DynamicEarth, a dedicated codebase designed to advance research and application of OVCD. https://likyoo.github.io/DynamicEarth

DynamicEarth: How Far are We from Open-Vocabulary Change Detection?

TL;DR

This work defines Open-Vocabulary Change Detection (OVCD) to localize and interpret changes between bi-temporal remote-sensing images for arbitrary categories described by text, addressing open-world requires no category-specific training. It introduces two training-free frameworks, M-C-I and I-M-C, that re-use off-the-shelf foundation models (e.g., SAM, DINO, CLIP, Grounding DINO) to detect, compare, and classify changes, avoiding extensive annotated data. Through experiments on multiple building and land-cover datasets, the authors show strong cross-dataset generalization and competitive performance against unsupervised and some supervised baselines, highlighting OVCD as a practical path toward open-world perception in Earth observation. The DynamicEarth codebase is released to enable researchers to reproduce results and extend OVCD methods in real-world scenarios.

Abstract

Monitoring Earth's evolving land covers requires methods capable of detecting changes across a wide range of categories and contexts. Existing change detection methods are hindered by their dependency on predefined classes, reducing their effectiveness in open-world applications. To address this issue, we introduce open-vocabulary change detection (OVCD), a novel task that bridges vision and language to detect changes across any category. Considering the lack of high-quality data and annotation, we propose two training-free frameworks, M-C-I and I-M-C, which leverage and integrate off-the-shelf foundation models for the OVCD task. The insight behind the M-C-I framework is to discover all potential changes and then classify these changes, while the insight of I-M-C framework is to identify all targets of interest and then determine whether their states have changed. Based on these two frameworks, we instantiate to obtain several methods, e.g., SAM-DINOv2-SegEarth-OV, Grounding-DINO-SAM2-DINO, etc. Extensive evaluations on 5 benchmark datasets demonstrate the superior generalization and robustness of our OVCD methods over existing supervised and unsupervised methods. To support continued exploration, we release DynamicEarth, a dedicated codebase designed to advance research and application of OVCD. https://likyoo.github.io/DynamicEarth
Paper Structure (11 sections, 1 equation, 3 figures, 3 tables)

This paper contains 11 sections, 1 equation, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Problem Definition
  4. Method
  5. M-C-I
  6. I-M-C
  7. Experiment
  8. Dataset
  9. Implementation Details
  10. Results
  11. Conclusion

Figures (3)

  • Figure 1: The two OVCD frameworks proposed in this paper. (a) M-C-I: discover all class-agnostic masks, determine if the mask region has changed, and identify the change class. (b) I-M-C: identify all targets of interest, convert to mask format, and compare if the target has changed.
  • Figure 2: Different change detection tasks: (a) Binary change detection aims at discovering all (interested) changes and generating a binary mask; (b) Semantic change detection further identifies the category of changes. However, both can only be trained and evaluated on data with predefined categories; (c) Our proposed OVCD can detect changes in any category according to the user's requirements.
  • Figure 3: Open-vocabulary change detection examples. In each group: $x_{{img}_1}$, $x_{{img}_2}$, ground truth, the result of an M-C-I method and the result of an I-M-C method. Color rendering: "Building", "Water", "Playground".