Zonal RL-RRT: Integrated RL-RRT Path Planning with Collision Probability and Zone Connectivity

TL;DR

A novel path-planning algorithm that leverages kd-tree partitioning to segment the map into zones while addressing zone connectivity, ensuring seamless transitions between zones, allowing the algorithm to accommodate flexible policies across diverse environments, making it a versatile tool for advanced path planning.

Abstract

Path planning in high-dimensional spaces poses significant challenges, particularly in achieving both time efficiency and a fair success rate. To address these issues, we introduce a novel path-planning algorithm, Zonal RL-RRT, that leverages kd-tree partitioning to segment the map into zones while addressing zone connectivity, ensuring seamless transitions between zones. By breaking down the complex environment into multiple zones and using Q-learning as the high-level decision-maker, our algorithm achieves a 3x improvement in time efficiency compared to basic sampling methods such as RRT and RRT* in forest-like maps. Our approach outperforms heuristic-guided methods like BIT* and Informed RRT* by 1.5x in terms of runtime while maintaining robust and reliable success rates across 2D to 6D environments. Compared to learning-based methods like NeuralRRT* and MPNetSMP, as well as the heuristic RRT*J, our algorithm demonstrates, on average, 1.5x better performance in the same environments. We also evaluate the effectiveness of our approach through simulations of the UR10e arm manipulator in the MuJoCo environment. A key observation of our approach lies in its use of zone partitioning and Reinforcement Learning (RL) for adaptive high-level planning allowing the algorithm to accommodate flexible policies across diverse environments, making it a versatile tool for advanced path planning.

Figures (5)

• Figure 1: Illustration of our Zonal RL-RRT approach. The input is the map including locations of all obstacles, start location, and goal positions. We partition the map into multiple zones using a kd-tree and establishes a state-action space accounting for the connectivity between zones. Our approach determines the optimal sequence of zones to traverse from the start to the goal. Finally, we utilize the Rapidly-exploring Random Trees (RRT) method low-level path planning within these zones and generate the final route.
• Figure 2: Comparative analysis of path planning algorithms in three different map scenarios. Top row: performance of baseline planners, including RRT*J (a), NRRT*-S2 (b), and MPNetSMP (c). Bottom row: performance of the proposed Zonal RL-RRT algorithm for the same maps (d, e, f).
• Figure 3: Comparative analysis of path planning algorithms on two 3D maps.
• Figure 4: An example of the sequence of configurations during the path planning process for a UR10e robot arm in the MuJoCo environment.
• Figure 5: Illustration of Zonal RL-RRT's flexibility with different rewards. In both maps, when $R_d \neq 0$, the agent takes a direct path, while $R_{\rho} \neq 0$ leads the agent around obstacles.