DDM-Lag : A Diffusion-based Decision-making Model for Autonomous Vehicles with Lagrangian Safety Enhancement

TL;DR

The paper addresses safe decision-making for autonomous vehicles in uncertain, dynamic environments by reframing sequential control as a generative diffusion process and enforcing safety via Lagrangian constraints. It introduces a diffusion-based decision policy (DDM-Lag) with a hybrid learning objective that combines behavior cloning and Q-learning within an Actor-Critic framework, and uses a PID-controlled Lagrange multiplier to regulate safety costs. A safety critic for constraint evaluation augments the learning process, and the method optimizes a Lagrangian objective $L(\theta,\lambda)=J^R(\pi_\theta)-\lambda(J^C(\pi_\theta)-d)$ to balance reward and safety. Experiments in the MetaDrive simulator show that DDM-Lag reduces safety violations while achieving strong driving performance across varying traffic densities, outperforming several baselines and demonstrating practical potential for safe autonomous driving. The work highlights a scalable approach to integrate diffusion-based decision-making with explicit safety guarantees and paves the way for further safety enhancements and efficiency improvements in AV systems.

Abstract

Decision-making stands as a pivotal component in the realm of autonomous vehicles (AVs), playing a crucial role in navigating the intricacies of autonomous driving. Amidst the evolving landscape of data-driven methodologies, enhancing decision-making performance in complex scenarios has emerged as a prominent research focus. Despite considerable advancements, current learning-based decision-making approaches exhibit potential for refinement, particularly in aspects of policy articulation and safety assurance. To address these challenges, we introduce DDM-Lag, a Diffusion Decision Model, augmented with Lagrangian-based safety enhancements. This work conceptualizes the sequential decision-making challenge inherent in autonomous driving as a problem of generative modeling, adopting diffusion models as the medium for assimilating patterns of decision-making. We introduce a hybrid policy update strategy for diffusion models, amalgamating the principles of behavior cloning and Q-learning, alongside the formulation of an Actor-Critic architecture for the facilitation of updates. To augment the model's exploration process with a layer of safety, we incorporate additional safety constraints, employing a sophisticated policy optimization technique predicated on Lagrangian relaxation to refine the policy learning endeavor comprehensively. Empirical evaluation of our proposed decision-making methodology was conducted across a spectrum of driving tasks, distinguished by their varying degrees of complexity and environmental contexts. The comparative analysis with established baseline methodologies elucidates our model's superior performance, particularly in dimensions of safety and holistic efficacy.

Figures (5)

• Figure 1: The overall procedure of our work.
• Figure 2: The conditioned diffusion process for our method.
• Figure 3: The framework of DDM-Lag algorithm.
• Figure 4: The three kinds of mixed scenario used for traning and testing our method, (a) scenario 1, (b) scenario 2, (c) mixed long-distance scenario 3.
• Figure 5: Interaction performance evaluation results for different scenarios, (a) scenario 1, (b) scenario 2.