Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Specialized Change Detection using Segment Anything

Tahir Ahmad, Sudipan Saha

TL;DR

The paper tackles specialized change detection, focusing on detecting the disappearance of specific objects (e.g., buildings) with very limited training data. It proposes a method that uses a pre-change object mask as a prompt to the Segment Anything Model (SAM) to analyze the post-change image and identify disappearances without training or access to the pre-change image. By exploiting SAM's promptable segmentation and confidence scores, the approach identifies changed objects by excluding candidate objects and selecting the exclusion that yields higher confidence, enabling extensions to multiple changes. Experimental validation on the SECOND dataset demonstrates capability for single and multi-object disappearances, with privacy-preserving and cross-sensor applicability benefits, while acknowledging challenges in complex scenes and the potential need for further development.

Abstract

Change detection (CD) is a fundamental task in Earth observation. While most change detection methods detect all changes, there is a growing need for specialized methods targeting specific changes relevant to particular applications while discarding the other changes. For instance, urban management might prioritize detecting the disappearance of buildings due to natural disasters or other reasons. Furthermore, while most supervised change detection methods require large-scale training datasets, in many applications only one or two training examples might be available instead of large datasets. Addressing such needs, we propose a focused CD approach using the Segment Anything Model (SAM), a versatile vision foundation model. Our method leverages a binary mask of the object of interest in pre-change images to detect their disappearance in post-change images. By using SAM's robust segmentation capabilities, we create prompts from the pre-change mask, use those prompts to segment the post-change image, and identify missing objects. This unsupervised approach demonstrated for building disappearance detection, is adaptable to various domains requiring specialized CD. Our contributions include defining a novel CD problem, proposing a method using SAM, and demonstrating its effectiveness. The proposed method also has benefits related to privacy preservation.

Specialized Change Detection using Segment Anything

TL;DR

The paper tackles specialized change detection, focusing on detecting the disappearance of specific objects (e.g., buildings) with very limited training data. It proposes a method that uses a pre-change object mask as a prompt to the Segment Anything Model (SAM) to analyze the post-change image and identify disappearances without training or access to the pre-change image. By exploiting SAM's promptable segmentation and confidence scores, the approach identifies changed objects by excluding candidate objects and selecting the exclusion that yields higher confidence, enabling extensions to multiple changes. Experimental validation on the SECOND dataset demonstrates capability for single and multi-object disappearances, with privacy-preserving and cross-sensor applicability benefits, while acknowledging challenges in complex scenes and the potential need for further development.

Abstract

Change detection (CD) is a fundamental task in Earth observation. While most change detection methods detect all changes, there is a growing need for specialized methods targeting specific changes relevant to particular applications while discarding the other changes. For instance, urban management might prioritize detecting the disappearance of buildings due to natural disasters or other reasons. Furthermore, while most supervised change detection methods require large-scale training datasets, in many applications only one or two training examples might be available instead of large datasets. Addressing such needs, we propose a focused CD approach using the Segment Anything Model (SAM), a versatile vision foundation model. Our method leverages a binary mask of the object of interest in pre-change images to detect their disappearance in post-change images. By using SAM's robust segmentation capabilities, we create prompts from the pre-change mask, use those prompts to segment the post-change image, and identify missing objects. This unsupervised approach demonstrated for building disappearance detection, is adaptable to various domains requiring specialized CD. Our contributions include defining a novel CD problem, proposing a method using SAM, and demonstrating its effectiveness. The proposed method also has benefits related to privacy preservation.
Paper Structure (14 sections, 5 figures)

This paper contains 14 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Related works
  3. Semantic segmentation
  4. Change detection
  5. Vision Transformers
  6. Foundation models
  7. Proposed Method
  8. Segment Anything
  9. Specialized change detection
  10. Experimental Validation
  11. Data
  12. Result for single disappeared object
  13. Result for multiple disappeared objects
  14. Conclusion

Figures (5)

  • Figure 1: Proposed data scenario: only pre-change mask is used by the proposed method and pre-change image itself is not required.
  • Figure 2: Proposed method
  • Figure 3: Result visualization on a scene with a single connected changed object: (a) Pre-change image, (b) Post-change image, (c) Reference, (d) Result obtained by the proposed method, (e) Result obtained by RCVA, (f) Result obtained by DCVA. Changed pixels are shown in white.
  • Figure 4: Another result visualization on a scene with a single connected changed object: (a) Pre-change image, (b) Post-change image, (c) Reference, (d) Result obtained by the proposed method.
  • Figure 5: Result visualization on a scene with a multiple connected changed object: (a) Pre-change image, (b) Post-change image, (c) Reference, (d) Result obtained by the proposed method considering single changed object, (e) Result obtained by the proposed method considering two changed objects.