When to Replan? An Adaptive Replanning Strategy for Autonomous Navigation using Deep Reinforcement Learning

TL;DR

This work tackles the problem of when to replan in hierarchical autonomous navigation to avoid stagnation or oscillations in partially unknown environments. It formulates replanning timing as a POMDP and introduces a DRL based replanner that learns environment specific strategies to trigger replanning. Through extensive simulations across multiple maps and planner combinations, the DRL replanner achieves robustness and efficiency on par with or better than traditional rule based strategies, and demonstrates reduced replanning frequency without sacrificing performance. The results suggest that learning adaptive replanning policies can significantly enhance navigation in dynamic, partially explored settings, with practical implications for integrating DRL into existing ROS 2 Navigation Stack deployments.

Abstract

The hierarchy of global and local planners is one of the most commonly utilized system designs in autonomous robot navigation. While the global planner generates a reference path from the current to goal locations based on the pre-built map, the local planner produces a kinodynamic trajectory to follow the reference path while avoiding perceived obstacles. To account for unforeseen or dynamic obstacles not present on the pre-built map, ``when to replan'' the reference path is critical for the success of safe and efficient navigation. However, determining the ideal timing to execute replanning in such partially unknown environments still remains an open question. In this work, we first conduct an extensive simulation experiment to compare several common replanning strategies and confirm that effective strategies are highly dependent on the environment as well as the global and local planners. Based on this insight, we then derive a new adaptive replanning strategy based on deep reinforcement learning, which can learn from experience to decide appropriate replanning timings in the given environment and planning setups. Our experimental results show that the proposed replanner can perform on par or even better than the current best-performing strategies in multiple situations regarding navigation robustness and efficiency.

Figures (5)

• Figure 1: Learning when to replan
• Figure 2: Proposed system overview
• Figure 4: Time transition according to the replanning action.
• Figure 5: A simulation result for 16-pillar map
• Figure 6: A simulation result for 25-pillar map