Multi-agent DRL-based Lane Change Decision Model for Cooperative Planning in Mixed Traffic

TL;DR

The paper tackles lane-change decision-making for connected automated vehicles in mixed traffic during the early deployment phase, where CAV density is low. It introduces a hybrid, multi-agent DRL framework that leverages CNN-QMIX to handle a dynamic number of agents and coordinates with a trajectory planner and MPC for safe, smooth execution. Key contributions include a grid-based CNN-QMIX architecture, a reward design that explicitly promotes cooperative platooning, and an MPC-backed execution layer; microsimulation results show improvements in platoon rates and energy efficiency, demonstrating robustness to varying market penetration rates. The work offers a scalable, real-time approach to enabling cooperative vehicular planning in realistic, fluctuating traffic conditions with potential impact on safety, energy savings, and traffic flow in early-stage CAV deployment.

Abstract

Connected automated vehicles (CAVs) possess the ability to communicate and coordinate with one another, enabling cooperative platooning that enhances both energy efficiency and traffic flow. However, during the initial stage of CAV deployment, the sparse distribution of CAVs among human-driven vehicles reduces the likelihood of forming effective cooperative platoons. To address this challenge, this study proposes a hybrid multi-agent lane change decision model aimed at increasing CAV participation in cooperative platooning and maximizing its associated benefits. The proposed model employs the QMIX framework, integrating traffic data processed through a convolutional neural network (CNN-QMIX). This architecture addresses a critical issue in dynamic traffic scenarios by enabling CAVs to make optimal decisions irrespective of the varying number of CAVs present in mixed traffic. Additionally, a trajectory planner and a model predictive controller are designed to ensure smooth and safe lane-change execution. The proposed model is trained and evaluated within a microsimulation environment under varying CAV market penetration rates. The results demonstrate that the proposed model efficiently manages fluctuating traffic agent numbers, significantly outperforming the baseline rule-based models. Notably, it enhances cooperative platooning rates up to 26.2\%, showcasing its potential to optimize CAV cooperation and traffic dynamics during the early stage of deployment.

Figures (6)

• Figure 1: Lane Change Decision for Speed Incentive and Sabotaging Cooperative Platooning
• Figure 2: Lane Change Process Overview
• Figure 3: The Architecture of the Multi-agent DRL-based Lane Change Decision Model
• Figure 4: The Input Process for the Multi-agent DRL-based Lane Change Decision Model
• Figure 5: Microsimulation Framework