AINav: Large Language Model-Based Adaptive Interactive Navigation

TL;DR

AINav tackles the challenge of navigating in unknown, interactive environments where no feasible path exists a priori. It combines an LLM-driven primitive skill tree for robust task planning, a reinforcement-learning-based skill library for robust motion and interaction, and an adaptive replanning mechanism with an advisor and arborist to rapidly adjust plans as new observations arrive. The approach enables proactive interaction with environmental objects to create viable routes, demonstrated through extensive simulations and real-world tests where the quadruped robot uses tools and adjusts plans in response to incremental information. The results show superior success rates and shorter completion times compared with baselines, highlighting the practical potential of integrating foundation-model reasoning with learned skills for long-horizon, tool-using robotic navigation in cluttered settings.

Abstract

Robotic navigation in complex environments remains a critical research challenge. Traditional navigation methods focus on optimal trajectory generation within fixed free workspace, therefore struggling in environments lacking viable paths to the goal, such as disaster zones or cluttered warehouses. To address this problem, we propose AINav, an adaptive interactive navigation approach that proactively interacts with environments to create feasible paths to achieve originally unreachable goals. Specifically, we present a primitive skill tree for task planning with large language models (LLMs), facilitating effective reasoning to determine interaction objects and sequences. To ensure robust subtask execution, we adopt reinforcement learning to pre-train a comprehensive skill library containing versatile locomotion and interaction behaviors for motion planning. Furthermore, we introduce an adaptive replanning approach featuring two LLM-based modules: an advisor serving as a flexible replanning trigger and an arborist for autonomous plan adjustment. Integrated with the tree structure, the replanning mechanism allows for convenient node addition and pruning, enabling rapid plan adaptation in a priori unknown environments. Comprehensive simulations and experiments have demonstrated AINav's effectiveness and adaptivity in diverse scenarios. The supplementary video is available at: https://youtu.be/CjXm5KFx9AI.

Figures (12)

• Figure 1: Interactive navigation in challenging environments. In contrast to conventional obstacle-avoidance navigation systems that passively adapt to predefined free space and fail to reach goals in environments without feasible paths, AINav can push aside obstacles to create pathways within congested obstacles or utilize obstacles as tools to traverse excessively high hurdles, enabling interactive navigation in challenging environments.
• Figure 2: An overview of AINav approach, a hierarchical system for interactive navigation. The task planning module processes visual input to generate a subtask skeleton for execution by the motion planning module, integrated with an adaptive replanning mechanism that enables flexible replan triggering and rapid adjustments to the task plan.
• Figure 3: Task planning with the proposer and the evaluator.
• Figure 4: Simulation environments for interactive navigation tasks.
• Figure 5: Average success rate and overall time across all scenarios of different methods.