BAP-SRL: Bayesian Adaptive Priority Safe Reinforcement Learning for Vehicle Motion Planning at Mixed Traffic Intersections

TL;DR

The paper tackles safe motion planning for autonomous vehicles at mixed-traffic intersections with multiple heterogeneous risk sources. It introduces Bayesian Adaptive Priority Safe Reinforcement Learning (BAP-SRL), which treats constraint priority as latent variables and gates policy gradients via Bayesian updates that combine historical optimization difficulty with instantaneous risk evidence. The method extends PPO-Lagrangian with a per-constraint adaptive weight $w_k(s_t,a_t)$ and decoupled reward and cost critics, reducing gradient cancellation and improving safety without sacrificing efficiency. Experiments in CARLA demonstrate lower collision rates, reduced average risk, and robust performance across adversarial scenarios, indicating practical impact for real-world mixed-traffic autonomy.

Abstract

Navigating urban intersections, especially when interacting with heterogeneous traffic participants, presents a formidable challenge for autonomous vehicles (AVs). In such environments, safety risks arise simultaneously from multiple sources, each carrying distinct priority levels and sensitivities that necessitate differential protection preferences. While safe reinforcement learning (RL) offers a robust paradigm for constrained decision-making, existing methods typically model safety as a single constraint or employ static, heuristic weighting schemes for multiple constraints. These approaches often fail to address the dynamic nature of multi-source risks, leading to gradient cancellation that hampers learning, and suboptimal trade-offs in critical dilemma zones. To address this, we propose a Bayesian adaptive priority safe reinforcement learning (BAP-SRL) based motion planning framework. Unlike heuristic weighting schemes, BAP formulates constraint prioritization as a probabilistic inference task. By modeling historical optimization difficulty as a Bayesian prior and instantaneous risk evidence as a likelihood, BAP dynamically gates gradient updates using a Bayesian inference mechanism on latent constraint criticality. Extensive experiments demonstrate that our approach outperforms state-of-the-art baselines in handling interactions with stochastic, heterogeneous agents, achieving lower collision rates and smoother conflict resolution.

Figures (7)

• Figure 1: The challenge of multi-objective conflict in safe autonomous driving. (a) The ego vehicle (blue) must navigate a dynamic intersection facing simultaneous collision threats from heterogeneous traffic agents, creating complex spatiotemporal conflicts. (b) An empirical analysis of constraint gradients during the training of the PPO-Lagrangian baseline ray2019benchmarking. By tracking the cosine similarity between gradients, we observe that some pairs frequently drop into the negative zone. This indicates physical conflicts translate into algorithmic gradient cancellation, potentially trapping the safe RL agent in behavioral dilemmas and optimization instability.
• Figure 2: The proposed BAP-SRL motion planning framework. The architecture is composed of three integrated modules: (1) Environment: the ego vehicle perceives the complex intersection environment to form state input while monitoring multiple safety constraints. (2) Actor-Critic Network: adopting a multi-critic architecture, the system utilizes a reward critic and independent cost critics to provide value feedback to the actor. The actor generates control actions and updates via a modifed gradient $g$. (3) BAP Mechanism: serving as a dynamic safety modulator, this module infers the gradient gating weight in the Lagrangian method by combining the latent prior belief and instantaneous risk evidence via Bayesian updates. The resulting adaptive weights $w_k$ perform sample-wise soft rescaling on cost gradients to dynamically prioritize critical safety violations.
• Figure 3: Illustration of the Bayesian adaptive priority mechanism. The framework models constraint prioritization as a probabilistic inference task, fusing prior belief (derived from static priority preferences and historical difficulty ) with likelihood evidence (combining immediate violation magnitude and predictive cost advantage). The final posterior weight is computed by Bayesian rule to dynamically gate the policy gradient.
• Figure 4: Learning curves during training. The solid lines represent the mean performance, and the shaded areas represent the standard deviation.
• Figure 5: Ablation analysis of static priority. (a) Collision rate breakdown by object type. The unbiased baseline results in a doubled total collision rate and significantly higher risk to VRUs. (b) Success rate across different maneuvers. The proposed BAP-SRL maintains high robustness, whereas the baseline performance drops drastically in complex right-turn scenarios.