A Reinforcement Learning-Boosted Motion Planning Framework: Comprehensive Generalization Performance in Autonomous Driving

TL;DR

The paper tackles the challenge of safe, adaptive motion planning for autonomous driving by integrating a sampling-based planner in Frenet coordinates with a reinforcement learning agent that learns to adjust trajectory-cost weights. The method leverages PPO with an LSTM to handle temporal dependencies, supported by a carefully designed observation and reward structure. Results show that the hybrid planner substantially reduces ego and road-user risk while maintaining real-time feasibility, and achieving zero collisions across unseen test scenarios in extensive evaluations. This hybrid approach offers improved generalization over purely analytical or purely learned methods and provides a practical framework with open-source software for real-world adoption.

Abstract

This study introduces a novel approach to autonomous motion planning, informing an analytical algorithm with a reinforcement learning (RL) agent within a Frenet coordinate system. The combination directly addresses the challenges of adaptability and safety in autonomous driving. Motion planning algorithms are essential for navigating dynamic and complex scenarios. Traditional methods, however, lack the flexibility required for unpredictable environments, whereas machine learning techniques, particularly reinforcement learning (RL), offer adaptability but suffer from instability and a lack of explainability. Our unique solution synergizes the predictability and stability of traditional motion planning algorithms with the dynamic adaptability of RL, resulting in a system that efficiently manages complex situations and adapts to changing environmental conditions. Evaluation of our integrated approach shows a significant reduction in collisions, improved risk management, and improved goal success rates across multiple scenarios. The code used in this research is publicly available as open-source software and can be accessed at the following link: https://github.com/TUM-AVS/Frenetix-RL.

Figures (10)

• Figure 1: Hybrid reinforcement learning principle of a motion planning agent. The agent's action informs an analytical (e.g., sampling-based) method to achieve the goal.
• Figure 2: Frenetix Motion Planner: sampling-based motion planning procedure for one timestep including prediction information and environment state update.
• Figure 3: Class diagram of the learning process structure.
• Figure 4: Trajectory cost observation space. The cost calculations of the outermost trajectories perceive additional cost information at each timestep.
• Figure 5: Mean ego and 3-party risks over various scenarios according to geisslingerconcept. Blue indicates the HP, and orange the DP.