Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeSy-Route: A Neuro-Symbolic Benchmark for Constrained Route Planning in Remote Sensing

Ming Yang, Zhi Zhou, Shi-Yu Tian, Kun-Yang Yu, Lan-Zhe Guo, Yu-Feng Li

Abstract

Remote sensing underpins crucial applications such as disaster relief and ecological field surveys, where systems must understand complex scenes and constraints and make reliable decisions. Current remote-sensing benchmarks mainly focus on evaluating perception and reasoning capabilities of multimodal large language models (MLLMs). They fail to assess planning capability, stemming either from the difficulty of curating and validating planning tasks at scale or from evaluation protocols that are inaccurate and inadequate. To address these limitations, we introduce NeSy-Route, a large-scale neuro-symbolic benchmark for constrained route planning in remote sensing. Within this benchmark, we introduce an automated data-generation framework that integrates high-fidelity semantic masks with heuristic search to produce diverse route-planning tasks with provably optimal solutions. This allows NeSy-Route to comprehensively evaluate planning across 10,821 route-planning samples, nearly 10 times larger than the largest prior benchmark. Furthermore, a three-level hierarchical neuro-symbolic evaluation protocol is developed to enable accurate assessment and support fine-grained analysis on perception, reasoning, and planning simultaneously. Our comprehensive evaluation of various state-of-the-art MLLMs demonstrates that existing MLLMs show significant deficiencies in perception and planning capabilities. We hope NeSy-Route can support further research and development of more powerful MLLMs for remote sensing.

NeSy-Route: A Neuro-Symbolic Benchmark for Constrained Route Planning in Remote Sensing

Abstract

Remote sensing underpins crucial applications such as disaster relief and ecological field surveys, where systems must understand complex scenes and constraints and make reliable decisions. Current remote-sensing benchmarks mainly focus on evaluating perception and reasoning capabilities of multimodal large language models (MLLMs). They fail to assess planning capability, stemming either from the difficulty of curating and validating planning tasks at scale or from evaluation protocols that are inaccurate and inadequate. To address these limitations, we introduce NeSy-Route, a large-scale neuro-symbolic benchmark for constrained route planning in remote sensing. Within this benchmark, we introduce an automated data-generation framework that integrates high-fidelity semantic masks with heuristic search to produce diverse route-planning tasks with provably optimal solutions. This allows NeSy-Route to comprehensively evaluate planning across 10,821 route-planning samples, nearly 10 times larger than the largest prior benchmark. Furthermore, a three-level hierarchical neuro-symbolic evaluation protocol is developed to enable accurate assessment and support fine-grained analysis on perception, reasoning, and planning simultaneously. Our comprehensive evaluation of various state-of-the-art MLLMs demonstrates that existing MLLMs show significant deficiencies in perception and planning capabilities. We hope NeSy-Route can support further research and development of more powerful MLLMs for remote sensing.
Paper Structure (30 sections, 16 equations, 2 figures, 6 tables)

This paper contains 30 sections, 16 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Remote Sensing Benchmark
  4. Multimodal Large Language Model
  5. Automated Data Generation Framework
  6. Knowledge Base Construction
  7. Problem Formulation
  8. Task 1: Textual Constraint Understanding
  9. Task 2: Text–Image Constraint Alignment
  10. Task 3: Constrained Route Planning
  11. Symbolized Data Generation
  12. Controllable Symbolic Query Synthesis
  13. Semantic Visual Grounding and Region Identification
  14. Constrained Trajectory Generation and Optimization
  15. NeSy-Route Benchmark
  16. ...and 15 more sections

Figures (2)

  • Figure 1: A typical example from our NeSy-Route Benchmark. NeSy-Route evaluates MLLMs through a hierarchical reasoning pipeline consisting of three integrated tasks. Task 1 involves extracting symbolic traversability and cost vectors from the provided hiker mission scenario. Task 2 focuses on anchoring these symbolic constraints to specific identified regions within the remote sensing image. Task 3 assesses the capability to generate a sparse waypoint trajectory that avoids obstacles and minimizes the cumulative cost.
  • Figure 2: An overview of automated data generation framework. The pipeline integrates dual-LLM logical verification for query synthesis, morphological erosion for semantic visual grounding, and constrained A-Star search algorithm for deriving mathematically optimal trajectories under symbolic rules.