A Nested Graph Reinforcement Learning-based Decision-making Strategy for Eco-platooning

TL;DR

The paper tackles virtual bottlenecks in large-scale mixed platooning caused by vehicle heterogeneity. It introduces a nested graph reinforcement learning framework (NSTW) that combines nested traffic graphs, a nested graphical MDP, and a multi-head nested graph attention network to model spatiotemporal interactions and drive multi-objective decisions. Empirical results on FLOW/I-24 scenarios show NSTW achieving about a 10% increase in throughput and a 9% reduction in energy consumption compared with IDM, with strong generalizability and clear benefits from higher CAV penetration, albeit with potential energy trade-offs. The work provides a scalable, energy-aware approach for coordinating heterogeneous vehicle fleets in real-world traffic, with implications for urban traffic management and autonomous mobility strategies.

Abstract

Platooning technology is renowned for its precise vehicle control, traffic flow optimization, and energy efficiency enhancement. However, in large-scale mixed platoons, vehicle heterogeneity and unpredictable traffic conditions lead to virtual bottlenecks. These bottlenecks result in reduced traffic throughput and increased energy consumption within the platoon. To address these challenges, we introduce a decision-making strategy based on nested graph reinforcement learning. This strategy improves collaborative decision-making, ensuring energy efficiency and alleviating congestion. We propose a theory of nested traffic graph representation that maps dynamic interactions between vehicles and platoons in non-Euclidean spaces. By incorporating spatio-temporal weighted graph into a multi-head attention mechanism, we further enhance the model's capacity to process both local and global data. Additionally, we have developed a nested graph reinforcement learning framework to enhance the self-iterative learning capabilities of platooning. Using the I-24 dataset, we designed and conducted comparative algorithm experiments, generalizability testing, and permeability ablation experiments, thereby validating the proposed strategy's effectiveness. Compared to the baseline, our strategy increases throughput by 10% and decreases energy use by 9%. Specifically, increasing the penetration rate of CAVs significantly enhances traffic throughput, though it also increases energy consumption.

Figures (18)

• Figure 1: The proposed framework is depicted in the schematic diagram, which employs a nested graph reinforcement learning-based decision-making model. Initially, mixed platooning scenarios are modeled using our developed nested traffic graph theory. This model is augmented by integrating the nested graph representation with a multi-head attention mechanism, resulting in a multi-head nested graph attention network module. Designed to capture the intricate spatio-temporal dependencies inherent in mixed platooning, this network aims for effective analysis. The decision-making process culminates with the generation of acceleration actions for CAVs through nested graph reinforcement learning. These actions are managed by a centralized control module, which orchestrates the CAVs within the platoon, thereby facilitating decisions that enhance safety, efficiency, comfort, and energy conservation.
• Figure 2: The hierarchical platoon architecture.
• Figure 3: A schematic diagram of a nested traffic graph.
• Figure 4: Distinction of non-isomorphic cyclic traffic graphs.
• Figure 5: Nested graphical markov decision process.

Theorems & Definitions (4)

• Definition 1: Nested traffic graph
• Theorem 1
• Theorem 2
• Definition 2: Nested graphical markov decision process