Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AstroReason-Bench: Evaluating Unified Agentic Planning across Heterogeneous Space Planning Problems

Weiyi Wang, Xinchi Chen, Jingjing Gong, Xuanjing Huang, Xipeng Qiu

TL;DR

AstroReason-Bench presents the first unified, physics-grounded benchmark suite for evaluating agentic planning across heterogeneous space planning problems. By integrating multiple subproblems under a common four-layer environment—physics, scenario management, interface, and cognitive agent—the work reveals both the capabilities and limitations of current agentic LLMs in long-horizon, constraint-rich domains. Across five benchmarks, state-of-the-art LLM agents outperform naïve baselines yet lag specialized solvers on tasks requiring exhaustive search, while demonstrating meaningful gains in constraint reasoning on more structure-driven problems. The benchmark provides a diagnostic platform to guide future development toward hybrid, knowledge-guided, and tool-augmented planning for space mission scheduling and resource management.

Abstract

Recent advances in agentic Large Language Models (LLMs) have positioned them as generalist planners capable of reasoning and acting across diverse tasks. However, existing agent benchmarks largely focus on symbolic or weakly grounded environments, leaving their performance in physics-constrained real-world domains underexplored. We introduce AstroReason-Bench, a comprehensive benchmark for evaluating agentic planning in Space Planning Problems (SPP), a family of high-stakes problems with heterogeneous objectives, strict physical constraints, and long-horizon decision-making. AstroReason-Bench integrates multiple scheduling regimes, including ground station communication and agile Earth observation, and provides a unified agent-oriented interaction protocol. Evaluating on a range of state-of-the-art open- and closed-source agentic LLM systems, we find that current agents substantially underperform specialized solvers, highlighting key limitations of generalist planning under realistic constraints. AstroReason-Bench offers a challenging and diagnostic testbed for future agentic research.

AstroReason-Bench: Evaluating Unified Agentic Planning across Heterogeneous Space Planning Problems

TL;DR

AstroReason-Bench presents the first unified, physics-grounded benchmark suite for evaluating agentic planning across heterogeneous space planning problems. By integrating multiple subproblems under a common four-layer environment—physics, scenario management, interface, and cognitive agent—the work reveals both the capabilities and limitations of current agentic LLMs in long-horizon, constraint-rich domains. Across five benchmarks, state-of-the-art LLM agents outperform naïve baselines yet lag specialized solvers on tasks requiring exhaustive search, while demonstrating meaningful gains in constraint reasoning on more structure-driven problems. The benchmark provides a diagnostic platform to guide future development toward hybrid, knowledge-guided, and tool-augmented planning for space mission scheduling and resource management.

Abstract

Recent advances in agentic Large Language Models (LLMs) have positioned them as generalist planners capable of reasoning and acting across diverse tasks. However, existing agent benchmarks largely focus on symbolic or weakly grounded environments, leaving their performance in physics-constrained real-world domains underexplored. We introduce AstroReason-Bench, a comprehensive benchmark for evaluating agentic planning in Space Planning Problems (SPP), a family of high-stakes problems with heterogeneous objectives, strict physical constraints, and long-horizon decision-making. AstroReason-Bench integrates multiple scheduling regimes, including ground station communication and agile Earth observation, and provides a unified agent-oriented interaction protocol. Evaluating on a range of state-of-the-art open- and closed-source agentic LLM systems, we find that current agents substantially underperform specialized solvers, highlighting key limitations of generalist planning under realistic constraints. AstroReason-Bench offers a challenging and diagnostic testbed for future agentic research.
Paper Structure (58 sections, 3 equations, 4 figures, 6 tables)

This paper contains 58 sections, 3 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. The Landscape of Satellite Planning and Scheduling
  4. Agentic Planning and Reasoning
  5. The AstroReason-Bench Suite
  6. Simulation Environment & Constraints
  7. Resource Constraints
  8. Kinematic Constraints
  9. Concurrency Constraints
  10. Benchmark Tasks
  11. Benchmark 1: SatNet (DSN Scheduling)
  12. Benchmark 2: Revisit Optimization
  13. Benchmark 3: Regional Coverage
  14. Benchmark 4: Stereo Imaging
  15. Benchmark 5: Latency-Optimization
  16. ...and 43 more sections

Figures (4)

  • Figure 1: Transition from disparate algorithms to a unified agentic framework: (a) illustrates the conventional methodology where tasks are isolated and optimized using disparate algorithms; (b) presents our unified agentic system, where a central intelligent agent leverages a toolkit to manage disparate scheduling tasks in an integrated manner.
  • Figure 2: The Environment and Interface Architecture. The architecture is organized into four layers: (1) The Physics Layer handles stateless physics computation; (2) The Scenario Layer manages session state; (3) The Interface Layer provides access to the environment via semantic MCP tools and a Python API; and (4) The Cognitive Layer hosts the LLM agent.
  • Figure 3: Single-Hop vs Multi-Hop Communication Strategies. (A) Failed approach: Agents attempt to find a single satellite simultaneously visible to both ground stations, which is geometrically impossible due to Earth's curvature. (B) Successful approach: Kat Coder Pro constructs a multi-hop ISL relay chain, enabling end-to-end connectivity across intercontinental distances.
  • Figure 4: Polygon Decomposition Strategies for Bay of Bengal. (A) Default mode: The agent registers 5 randomly-oriented strips without querying satellite ground tracks, resulting in limited access windows. (B) Plan mode: The agent correctly reasons about near-polar orbits and produces N-S aligned strips, improving coverage efficiency.