Path Planning through Multi-Agent Reinforcement Learning in Dynamic Environments
Jonas De Maeyer, Hossein Yarahmadi, Moharram Challenger
TL;DR
This work tackles dynamic path planning by introducing a region-aware, hierarchical reinforcement learning framework that eliminates reliance on a global planner. It deploys leaf-level Q-learning within a tree-structured decomposition of the environment and augments this with a retraining condition based on sub-environment success rates, plus federated Q-learning to accelerate learning. The key contributions include the hierarchical decomposition, a region-aware retraining mechanism, and two federated learning variants (EqAvg and ImAvg) that outperform single-agent baselines and approach A* Oracle in accuracy while offering better adaptation times and scalability. Experiments across easy, medium, and hard mazes, as well as a rigorous edge-case, demonstrate that federated approaches achieve robust performance with lower initial training and cumulative adaptation costs, supporting scalable deployment in dynamic, real-world settings. The study also outlines limitations and future work, including deep RL extensions, improved path optimality, and real-world robotic integration.
Abstract
Path planning in dynamic environments is a fundamental challenge in intelligent transportation and robotics, where obstacles and conditions change over time, introducing uncertainty and requiring continuous adaptation. While existing approaches often assume complete environmental unpredictability or rely on global planners, these assumptions limit scalability and practical deployment in real-world settings. In this paper, we propose a scalable, region-aware reinforcement learning (RL) framework for path planning in dynamic environments. Our method builds on the observation that environmental changes, although dynamic, are often localized within bounded regions. To exploit this, we introduce a hierarchical decomposition of the environment and deploy distributed RL agents that adapt to changes locally. We further propose a retraining mechanism based on sub-environment success rates to determine when policy updates are necessary. Two training paradigms are explored: single-agent Q-learning and multi-agent federated Q-learning, where local Q-tables are aggregated periodically to accelerate the learning process. Unlike prior work, we evaluate our methods in more realistic settings, where multiple simultaneous obstacle changes and increasing difficulty levels are present. Results show that the federated variants consistently outperform their single-agent counterparts and closely approach the performance of A* Oracle while maintaining shorter adaptation times and robust scalability. Although initial training remains time-consuming in large environments, our decentralized framework eliminates the need for a global planner and lays the groundwork for future improvements using deep RL and flexible environment decomposition.