Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReasonCD: A Multimodal Reasoning Large Model for Implicit Change-of-Interest Semantic Mining

Zhenyang Huang, Xiao Yu, Yi Zhang, Decheng Wang, Hang Ruan

TL;DR

<3-5 sentence high-level summary>

Abstract

Remote sensing image change detection is one of the fundamental tasks in remote sensing intelligent interpretation. Its core objective is to identify changes within change regions of interest (CRoI). Current multimodal large models encode rich human semantic knowledge, which is utilized for guidance in tasks such as remote sensing change detection. However, existing methods that use semantic guidance for detecting users' CRoI overly rely on explicit textual descriptions of CRoI, leading to the problem of near-complete performance failure when presented with implicit CRoI textual descriptions. This paper proposes a multimodal reasoning change detection model named ReasonCD, capable of mining users' implicit task intent. The model leverages the powerful reasoning capabilities of pre-trained large language models to mine users' implicit task intents and subsequently obtains different change detection results based on these intents. Experiments on public datasets demonstrate that the model achieves excellent change detection performance, with an F1 score of 92.1\% on the BCDD dataset. Furthermore, to validate its superior reasoning functionality, this paper annotates a subset of reasoning data based on the SECOND dataset. Experimental results show that the model not only excels at basic reasoning-based change detection tasks but can also explain the reasoning process to aid human decision-making.

ReasonCD: A Multimodal Reasoning Large Model for Implicit Change-of-Interest Semantic Mining

TL;DR

<3-5 sentence high-level summary>

Abstract

Remote sensing image change detection is one of the fundamental tasks in remote sensing intelligent interpretation. Its core objective is to identify changes within change regions of interest (CRoI). Current multimodal large models encode rich human semantic knowledge, which is utilized for guidance in tasks such as remote sensing change detection. However, existing methods that use semantic guidance for detecting users' CRoI overly rely on explicit textual descriptions of CRoI, leading to the problem of near-complete performance failure when presented with implicit CRoI textual descriptions. This paper proposes a multimodal reasoning change detection model named ReasonCD, capable of mining users' implicit task intent. The model leverages the powerful reasoning capabilities of pre-trained large language models to mine users' implicit task intents and subsequently obtains different change detection results based on these intents. Experiments on public datasets demonstrate that the model achieves excellent change detection performance, with an F1 score of 92.1\% on the BCDD dataset. Furthermore, to validate its superior reasoning functionality, this paper annotates a subset of reasoning data based on the SECOND dataset. Experimental results show that the model not only excels at basic reasoning-based change detection tasks but can also explain the reasoning process to aid human decision-making.
Paper Structure (36 sections, 24 equations, 17 figures, 4 tables)

This paper contains 36 sections, 24 equations, 17 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Semantics-First Change Detection Methods
  4. Multimodal Reasoning Large Models
  5. Methodology
  6. Problem Definition
  7. The ReasonCD Framework
  8. Change Detection Task Text Modeling
  9. Text Tokenization and Embedding
  10. Bitemporal Image Embedding
  11. Multimodal Token Reasoning
  12. Change Map Decoding Based on the $<CHG>$ Token
  13. Change Detection Task Text Modeling
  14. Text Tokenization and Embedding
  15. Bitemporal Image Embedding
  16. ...and 21 more sections

Figures (17)

  • Figure 1: Explicit and Implicit Semantic Expressions of CRoI, Using Buildings as an Example.
  • Figure 2: Schematic diagram of the overall ReasonCD structure. ReasonCD is mainly divided into five parts: (1) Change detection task text modeling; (2) Text tokenization and embedding; (3) Bitemporal image embedding; (4) Multimodal token reasoning; (5) Change map decoding based on the $<CHG>$ token.
  • Figure 3: Schematic diagram of CLIP principle
  • Figure 4: Comparison of Transformer architectures between (a) the standard Transformer and (b) LLaMA2.
  • Figure 5: Comparison diagram of (a) MHA and (b) GQA structures.
  • ...and 12 more figures