Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing

Xuran Hu, Zhitong Xiong, Zhongcheng Hong, Yifang Ban, Xiaoxiang Zhu, Wufan Zhao

Abstract

Current Large Multimodal Models (LMMs) in Earth Observation typically neglect the critical "vertical" dimension, limiting their reasoning capabilities in complex remote sensing geometries and disaster scenarios where physical spatial structures often outweigh planar visual textures. To bridge this gap, we introduce a comprehensive evaluation framework dedicated to height-aware remote sensing understanding. First, to overcome the severe scarcity of annotated data, we develop a scalable, VLM-driven data generation pipeline utilizing systematic prompt engineering and metadata extraction. This pipeline constructs two complementary benchmarks: GeoHeight-Bench for relative height analysis, and a more challenging GeoHeight-Bench+ for holistic, terrain-aware reasoning. Furthermore, to validate the necessity of height perception, we propose GeoHeightChat, the first height-aware remote sensing LMM baseline. Serving as a strong proof of concept, our baseline demonstrates that synergizing visual semantics with implicitly injected height geometric features effectively mitigates the "vertical blind spot", successfully unlocking a new paradigm of interactive height reasoning in existing optical models.

GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing

Abstract

Current Large Multimodal Models (LMMs) in Earth Observation typically neglect the critical "vertical" dimension, limiting their reasoning capabilities in complex remote sensing geometries and disaster scenarios where physical spatial structures often outweigh planar visual textures. To bridge this gap, we introduce a comprehensive evaluation framework dedicated to height-aware remote sensing understanding. First, to overcome the severe scarcity of annotated data, we develop a scalable, VLM-driven data generation pipeline utilizing systematic prompt engineering and metadata extraction. This pipeline constructs two complementary benchmarks: GeoHeight-Bench for relative height analysis, and a more challenging GeoHeight-Bench+ for holistic, terrain-aware reasoning. Furthermore, to validate the necessity of height perception, we propose GeoHeightChat, the first height-aware remote sensing LMM baseline. Serving as a strong proof of concept, our baseline demonstrates that synergizing visual semantics with implicitly injected height geometric features effectively mitigates the "vertical blind spot", successfully unlocking a new paradigm of interactive height reasoning in existing optical models.

Paper Structure

This paper contains 19 sections, 5 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Multimodal Models in Remote Sensing
  4. Reasoning Segmentation and Agents
  5. Remote Sensing Benchmarks and Height Perception
  6. GeoHeight-Bench
  7. Benchmark Tasks
  8. Benchmark Pipeline
  9. GeoHeightChat
  10. Problem Formulation
  11. Architecture Overview
  12. Cross-Modal Geo-Alignment
  13. Geo-Aware Instruction Tuning
  14. Experimental Results
  15. Experiment Setting
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Overview of the GeoHeight-Bench (+). We construct a comprehensive benchmark comprising 10 diverse tasks organized into four hierarchical levels: Pixel-level retrieval, Object-level extraction, Scene-level analysis, and Reasoning-level inference.
  • Figure 2: Pipeline of the GeoHeight-Bench generation and verification.
  • Figure 3: The proposed GeiHeightChat framework comprises two training stages: Cross-Modal Geo-Alignment and Geo-Aware Instruction Tuning.
  • Figure 4: Comparison between GeoHeightChat and LMMs.