Dyna-Style Learning with A Macroscopic Model for Vehicle Platooning in Mixed-Autonomy Traffic

TL;DR

The paper tackles fuel-efficient platooning in mixed-autonomy highway traffic by coupling a macroscopic LWR-based PDE with a platoon-tracking ODE and augmenting it with Kalman updates to adapt density and speed parameters. It introduces a Dyna-style planning and learning framework that uses this interpretable PDE-ODE model as a world model to generate virtual experiences for Q-learning, aiming to minimize fuel consumption. Empirical results in SUMO show a roughly 10% reduction in total fuel use and faster convergence than model-free baselines, alongside improved congestion metrics at bottlenecks. The approach offers a transparent, data-efficient alternative to neural-network heavy models and demonstrates practical impact for highway efficiency in mixed autonomy scenarios.

Abstract

Platooning of connected and autonomous vehicles (CAVs) plays a vital role in modernizing highways, ushering in enhanced efficiency and safety. This paper explores the significance of platooning in smart highways, employing a coupled partial differential equation (PDE) and ordinary differential equation (ODE) model to elucidate the complex interaction between bulk traffic flow and CAV platoons. Our study focuses on developing a Dyna-style planning and learning framework tailored for platoon control, with a specific goal of reducing fuel consumption. By harnessing the coupled PDE-ODE model, we improve data efficiency in Dyna-style learning through virtual experiences. Simulation results validate the effectiveness of our macroscopic model in modeling platoons within mixed-autonomy settings, demonstrating a notable $10.11\%$ reduction in vehicular fuel consumption compared to conventional approaches.

Figures (6)

• Figure 1: The mixed-autonomy traffic through a bottleneck on highway segment.
• Figure 2: The road section is divided into three sections: a preheating section, a three-lane section, and a two-lane section. The bottleneck occurs at the junction between the three-lane and two-lane sections.
• Figure 3: Average speed prediction errors for all cells using Kalman filter-trained parameters and untrained parameters.
• Figure 4: A comparison between the reward curves of the DQN and Dyna-Q methods, where Dyna-Q incorporates the macroscopic model.
• Figure 5: Fuel consumption rates throughout the simulation under the Krauss model and Dyna-Q.