Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Think and Answer ME: Benchmarking and Exploring Multi-Entity Reasoning Grounding in Remote Sensing

Shuchang Lyu, Haiquan Wen, Guangliang Cheng, Meng Li, Zheng Zhou, You Zhou, Dingding Yao, Zhenwei Shi

Abstract

Recent advances in reasoning language models and reinforcement learning with verifiable rewards have significantly enhanced multi-step reasoning capabilities. This progress motivates the extension of reasoning paradigms to remote sensing visual grounding task. However, existing remote sensing grounding methods remain largely confined to perception-level matching and single-entity formulations, limiting the role of explicit reasoning and inter-entity modeling. To address this challenge, we introduce a new benchmark dataset for Multi-Entity Reasoning Grounding in Remote Sensing (ME-RSRG). Based on ME-RSRG, we reformulate remote sensing grounding as a multi-entity reasoning task and propose an Entity-Aware Reasoning (EAR) framework built upon visual-linguistic foundation models. EAR generates structured reasoning traces and subject-object grounding outputs. It adopts supervised fine-tuning for cold-start initialization and is further optimized via entity-aware reward-driven Group Relative Policy Optimization (GRPO). Extensive experiments on ME-RSRG demonstrate the challenges of multi-entity reasoning and verify the effectiveness of our proposed EAR framework. Our dataset, code, and models will be available at https://github.com/CV-ShuchangLyu/ME-RSRG.

Think and Answer ME: Benchmarking and Exploring Multi-Entity Reasoning Grounding in Remote Sensing

Abstract

Recent advances in reasoning language models and reinforcement learning with verifiable rewards have significantly enhanced multi-step reasoning capabilities. This progress motivates the extension of reasoning paradigms to remote sensing visual grounding task. However, existing remote sensing grounding methods remain largely confined to perception-level matching and single-entity formulations, limiting the role of explicit reasoning and inter-entity modeling. To address this challenge, we introduce a new benchmark dataset for Multi-Entity Reasoning Grounding in Remote Sensing (ME-RSRG). Based on ME-RSRG, we reformulate remote sensing grounding as a multi-entity reasoning task and propose an Entity-Aware Reasoning (EAR) framework built upon visual-linguistic foundation models. EAR generates structured reasoning traces and subject-object grounding outputs. It adopts supervised fine-tuning for cold-start initialization and is further optimized via entity-aware reward-driven Group Relative Policy Optimization (GRPO). Extensive experiments on ME-RSRG demonstrate the challenges of multi-entity reasoning and verify the effectiveness of our proposed EAR framework. Our dataset, code, and models will be available at https://github.com/CV-ShuchangLyu/ME-RSRG.
Paper Structure (29 sections, 6 equations, 9 figures, 5 tables)

This paper contains 29 sections, 6 equations, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Visual Grounding in Remote Sensing
  4. Visual-Linguistic Reasoning Models
  5. Visual-Linguistic Reasoning in Remote Sensing
  6. Proposed Dataset: ME-RSRG
  7. Dataset Production
  8. Dataset Analysis
  9. Methodology
  10. Overall Architecture
  11. Entity-Aware Reward-Driven GRPO
  12. Optimization
  13. Experimental Results
  14. Experimental Setup
  15. Main Experimental Results
  16. ...and 14 more sections

Figures (9)

  • Figure 1: Paradigm comparison between (a) matching-based visual grounding and (b) multi-entity reasoning grounding in remote sensing.
  • Figure 2: Overview of ME-RSRG dataset. It includes (a) Dataset production, (b) Dataset statistics, and (c) Data case.
  • Figure 3: Overview of EAR framework. We adopt a two-stage optimization strategy: SFT initialization and entity-aware reward-driven GRPO refinement.
  • Figure 4: Qualitative Results Visualization.
  • Figure 5: Failure Cases Visualization.
  • ...and 4 more figures