Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid Motion Planning with Deep Reinforcement Learning for Mobile Robot Navigation

Yury Kolomeytsev, Dmitry Golembiovsky

TL;DR

HMP-DRL presents a hybrid navigation framework that combines a graph-based global planner with a semantically aware local DRL policy to tackle long-range, socially compliant robot navigation in dynamic urban environments. By converting the global path into a sequence of checkpoints and augmenting the local policy with checkpoint features and entity-type aware penalties, the approach achieves superior success rates, reduced collisions, and faster travel times compared to state-of-the-art baselines. The method demonstrates robust performance in a large-scale urban simulator derived from OpenStreetMap data, and ablation studies confirm the importance of checkpoint guidance for aligning local decisions with global routes. This hybrid architecture offers a practical pathway to reliable and safe autonomous navigation in human-centric settings, with clear directions for real-world extension and deployment.

Abstract

Autonomous mobile robots operating in complex, dynamic environments face the dual challenge of navigating large-scale, structurally diverse spaces with static obstacles while safely interacting with various moving agents. Traditional graph-based planners excel at long-range pathfinding but lack reactivity, while Deep Reinforcement Learning (DRL) methods demonstrate strong collision avoidance but often fail to reach distant goals due to a lack of global context. We propose Hybrid Motion Planning with Deep Reinforcement Learning (HMP-DRL), a hybrid framework that bridges this gap. Our approach utilizes a graph-based global planner to generate a path, which is integrated into a local DRL policy via a sequence of checkpoints encoded in both the state space and reward function. To ensure social compliance, the local planner employs an entity-aware reward structure that dynamically adjusts safety margins and penalties based on the semantic type of surrounding agents. We validate the proposed method through extensive testing in a realistic simulation environment derived from real-world map data. Comprehensive experiments demonstrate that HMP-DRL consistently outperforms other methods, including state-of-the-art approaches, in terms of key metrics of robot navigation: success rate, collision rate, and time to reach the goal. Overall, these findings confirm that integrating long-term path guidance with semantically-aware local control significantly enhances both the safety and reliability of autonomous navigation in complex human-centric settings.

Hybrid Motion Planning with Deep Reinforcement Learning for Mobile Robot Navigation

TL;DR

HMP-DRL presents a hybrid navigation framework that combines a graph-based global planner with a semantically aware local DRL policy to tackle long-range, socially compliant robot navigation in dynamic urban environments. By converting the global path into a sequence of checkpoints and augmenting the local policy with checkpoint features and entity-type aware penalties, the approach achieves superior success rates, reduced collisions, and faster travel times compared to state-of-the-art baselines. The method demonstrates robust performance in a large-scale urban simulator derived from OpenStreetMap data, and ablation studies confirm the importance of checkpoint guidance for aligning local decisions with global routes. This hybrid architecture offers a practical pathway to reliable and safe autonomous navigation in human-centric settings, with clear directions for real-world extension and deployment.

Abstract

Autonomous mobile robots operating in complex, dynamic environments face the dual challenge of navigating large-scale, structurally diverse spaces with static obstacles while safely interacting with various moving agents. Traditional graph-based planners excel at long-range pathfinding but lack reactivity, while Deep Reinforcement Learning (DRL) methods demonstrate strong collision avoidance but often fail to reach distant goals due to a lack of global context. We propose Hybrid Motion Planning with Deep Reinforcement Learning (HMP-DRL), a hybrid framework that bridges this gap. Our approach utilizes a graph-based global planner to generate a path, which is integrated into a local DRL policy via a sequence of checkpoints encoded in both the state space and reward function. To ensure social compliance, the local planner employs an entity-aware reward structure that dynamically adjusts safety margins and penalties based on the semantic type of surrounding agents. We validate the proposed method through extensive testing in a realistic simulation environment derived from real-world map data. Comprehensive experiments demonstrate that HMP-DRL consistently outperforms other methods, including state-of-the-art approaches, in terms of key metrics of robot navigation: success rate, collision rate, and time to reach the goal. Overall, these findings confirm that integrating long-term path guidance with semantically-aware local control significantly enhances both the safety and reliability of autonomous navigation in complex human-centric settings.
Paper Structure (30 sections, 27 equations, 4 figures, 3 tables, 3 algorithms)

This paper contains 30 sections, 27 equations, 4 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Proposed Approach
  5. Global Planning
  6. Graph Construction Methods.
  7. Search Algorithm.
  8. Experimental Setup.
  9. Local Planning
  10. Reward Function
  11. Model
  12. Checkpoint Feature Extraction.
  13. Extended Self-State.
  14. Entity State Embedding.
  15. Social Attentive Pooling.
  16. ...and 15 more sections

Figures (4)

  • Figure 1: Three examples of global paths constructed by the A* algorithm for different start and goal pairs in our experiments. Occupied cells are marked in gray, which in this episode represent residential and non-residential buildings of various sizes and shapes in the city.
  • Figure 2: Simulator snapshot during the simulation of the HMP-DRL method. The blue polyline is the global path. Green circles are checkpoints. Gray objects are residential and non-residential buildings in the city. The remaining objects are the robot and the dynamic agents surrounding it.
  • Figure 3: Example of map processing: (a) an OSM map snapshot from which occupancy maps were formed; (b) the corresponding binary occupancy map with a global path constructed using A*.
  • Figure 4: Collision rate, success rate, reward, weighted score metrics on the training and validation data. The validation step is performed every 1024 episodes.