A Multi-Agent Rollout Approach for Highway Bottleneck Decongestion in Mixed Autonomy

TL;DR

The paper tackles bottleneck congestion in mixed autonomy traffic by coordinating autonomous vehicle speeds through a Dec-POMDP formulation. It introduces a multi-agent rollout framework based on agent-by-agent policy iteration (A2PI) and integrates Proximal Policy Optimization (PPO) to refine policies sequentially, ensuring monotonic improvements and stability. Empirical validation on the Shenhai Highway using SUMO shows notable gains, including a 9.42% average travel-time reduction at 10% AV penetration and up to 45.13% reductions in congested scenarios, outperforming baselines MATD3 and MAPPO. The approach demonstrates robustness to varying agent counts and demonstrates a practical path toward real-time bottleneck decongestion in mixed autonomy networks.

Abstract

The integration of autonomous vehicles (AVs) into the existing transportation infrastructure offers a promising solution to alleviate congestion and enhance mobility. This research explores a novel approach to traffic optimization by employing a multi-agent rollout approach within a mixed autonomy environment. The study concentrates on coordinating the speed of human-driven vehicles by longitudinally controlling AVs, aiming to dynamically optimize traffic flow and alleviate congestion at highway bottlenecks in real-time. We model the problem as a decentralized partially observable Markov decision process (Dec-POMDP) and propose an improved multi-agent rollout algorithm. By employing agent-by-agent policy iterations, our approach implicitly considers cooperation among multiple agents and seamlessly adapts to complex scenarios where the number of agents dynamically varies. Validated in a real-world network with varying AV penetration rates and traffic flow, the simulations demonstrate that the multi-agent rollout algorithm significantly enhances performance, reducing average travel time on bottleneck segments by 9.42% with a 10% AV penetration rate.

Figures (6)

• Figure 1: Illustration of congestion where three lanes merge into two, with congestion originating downstream (right) and propagating upstream (left).
• Figure 2: Sequential policy update in multi-agent decentralized control.
• Figure 3: Traffic flow optimization at a bottleneck on the Shenhai Highway.
• Figure 4: The performance curves of the algorithms during training in the first scenario. MARollout (iter.1) and MARollout (iter.2) represent the results of 1-step and 2-step iteration, using the multi-agent rollout approach, respectively.
• Figure 5: An illustration of the cooperative actions of AVs near bottlenecks.