Trajectory Planning Using Reinforcement Learning for Interactive Overtaking Maneuvers in Autonomous Racing Scenarios

TL;DR

The paper tackles interactive overtaking in autonomous racing by showing that conventional prediction-based planning struggles when opponents react. It introduces an RL-based trajectory planner that directly generates end states and exploits inter-vehicle interaction, paired with a Safety Layer to recover feasible trajectories when necessary. The approach demonstrates higher success rates and orders-of-magnitude faster planning times than the conventional method, particularly under aggressive blocking, while highlighting generalization limitations and the complementary role of safety mechanisms. This work provides a practical framework for interaction-aware planning in racing and points to future extensions to richer scenarios and sensing modalities.

Abstract

Conventional trajectory planning approaches for autonomous racing are based on the sequential execution of prediction of the opposing vehicles and subsequent trajectory planning for the ego vehicle. If the opposing vehicles do not react to the ego vehicle, they can be predicted accurately. However, if there is interaction between the vehicles, the prediction loses its validity. For high interaction, instead of a planning approach that reacts exclusively to the fixed prediction, a trajectory planning approach is required that incorporates the interaction with the opposing vehicles. This paper demonstrates the limitations of a widely used conventional sampling-based approach within a highly interactive blocking scenario. We show that high success rates are achieved for less aggressive blocking behavior but that the collision rate increases with more significant interaction. We further propose a novel Reinforcement Learning (RL)-based trajectory planning approach for racing that explicitly exploits the interaction with the opposing vehicle without requiring a prediction. In contrast to the conventional approach, the RL-based approach achieves high success rates even for aggressive blocking behavior. Furthermore, we propose a novel safety layer (SL) that intervenes when the trajectory generated by the RL-based approach is infeasible. In that event, the SL generates a sub-optimal but feasible trajectory, avoiding termination of the scenario due to a not found valid solution.

Figures (7)

• Figure 1: Initialization of the blocking scenario with the overtaking vehicle (blue) at position $(s_\mathrm{o, init} = 0m, n_\mathrm{o, init} = 0m)$ and the blocking vehicle (green) at $(s_\mathrm{b, init}, n_\mathrm{b, init})$. The reference line of the considered straight race track (not to scale) is depicted as a dashed black line.
• Figure 2: Step responses for different values of $s_{\mathrm{d}}$ with $k_{\mathrm{p}}=0.05$, $k_{\mathrm{d}}=0.6$, and $k_{\mathrm{n}}=1.0$. The aggressiveness of the blocking maneuver increases with lower values of $s_{\mathrm{d}}$.
• Figure 3: Elliptical prediction cost shape for different parameterizations. The costs range from $d_{\mathrm{pr}}=0.0$ (blue) to $d_{\mathrm{pr}}=1.0$ (orange). The actual vehicle geometry is depicted in black. From left to right: small ellipse $(p_{\mathrm{s}}=0.08, p_{\mathrm{n}}=0.5)$, medium ellipse $(p_{\mathrm{s}}=0.02, p_{\mathrm{n}}=0.18)$, and large ellipse $(p_{\mathrm{s}}=0.01, p_{\mathrm{n}}=0.1)$.
• Figure 4: Evaluation of the conventional approach with the parameterizations listed in Table \ref{['tab:cost_parameters']} for different blocking behaviors specified by $s_{\mathrm{d}}$. The terms small, medium (med.), and large correspond to the different ellipse sizes.
• Figure 5: Evaluation of the RL-based approach with the training parameterizations listed in Table \ref{['tab:evaluation_training_parameters']} for different blocking behaviors specified by $s_{\mathrm{d}}$.