USS-Nav: Unified Spatio-Semantic Scene Graph for Lightweight UAV Zero-Shot Object Navigation

TL;DR

USS-Nav introduces a lightweight, onboard framework that incrementally builds a Unified Spatio-Semantic Scene Graph for UAV zero-shot object navigation. It fuses a Global Spatial Connectivity Graph, topological region partitioning, and online object instantiation with an LLM-driven coarse-to-fine planner, achieving real-time performance on edge hardware at $15$ Hz. The approach demonstrates substantial gains in navigation efficiency and robustness through hierarchical decision making, validated in high-fidelity simulations and real-world deployments, with ablations confirming the value of semantic grounding. By providing a persistent, open-vocabulary scene representation, USS-Nav enables practical semantic exploration and downstream reasoning on constrained aerial platforms, with public code to foster further research.

Abstract

Zero-Shot Object Navigation in unknown environments poses significant challenges for Unmanned Aerial Vehicles (UAVs) due to the conflict between high-level semantic reasoning requirements and limited onboard computational resources. To address this, we present USS-Nav, a lightweight framework that incrementally constructs a Unified Spatio-Semantic scene graph and enables efficient Large Language Model (LLM)-augmented Zero-Shot Object Navigation in unknown environments. Specifically, we introduce an incremental Spatial Connectivity Graph generation method utilizing polyhedral expansion to capture global geometric topology, which is dynamically partitioned into semantic regions via graph clustering. Concurrently, open-vocabulary object semantics are instantiated and anchored to this topology to form a hierarchical environmental representation. Leveraging this hierarchical structure, we present a coarse-to-fine exploration strategy: LLM grounded in the scene graph's semantics to determine global target regions, while a local planner optimizes frontier coverage based on information gain. Experimental results demonstrate that our framework outperforms state-of-the-art methods in terms of computational efficiency and real-time update frequency (15 Hz) on a resource-constrained platform. Furthermore, ablation studies confirm the effectiveness of our framework, showing substantial improvements in Success weighted by Path Length (SPL). The source code will be made publicly available to foster further research.

Figures (7)

• Figure 1: Overview of the proposed framework deployed on a resource-constrained UAV for Zero-Shot Object Navigation in a real-world environment. The system incrementally constructs a Global Spatial Connectivity Graph visualized as a wireframe and concurrently instantiates semantic objects to form a unified spatio-semantic scene graph. This hierarchical representation supports dynamic region partitioning, enabling the Large Language Model for efficient, coarse-to-fine decision making.
• Figure 2: Illustration of the incremental Spatial Connectivity Graph generation pipeline. Sub-figures (a) through (d) sequentially demonstrate the algorithmic workflow, evolving from uniform spherical sampling and polyhedral boundary expansion to the final topological update. The process constructs the Global Spatial Connectivity Graph directly from the current local occupancy grid map.
• Figure 3: Demonstration of the proposed spatial representation. This visualization showcases the effective polyhedral expansion for covering free space and the resulting region partitioning. It illustrates the dynamic process where newly observed nodes are seamlessly fused into established regions, ensuring topological consistency.
• Figure 4: Framework for associating object semantics with spatial information in the scene graph, leveraging advanced open-vocabulary segmentation models combined with spatial and semantic similarity measures. (a) Ground truth in the simulation environment. (b) Object processing results.
• Figure 5: Frontier generation and regional association. (a) Frontier generation utilizing the Global Coverage Mask. White regions indicate visited status, suppressing subsequent frontier generation in these areas. (b) After the LLM-based coarse-level global reasoning, a TSP solver is employed within the target region for fine-level local planning.