Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SOC-Boundary and Battery Aging Aware Hierarchical Coordination of Multiple EV Aggregates Among Multi-stakeholders with Multi-Agent Constrained Deep Reinforcement Learning

Xin Chen

TL;DR

The paper tackles the challenge of coordinating large-scale EVs for vehicle-to-grid (V2G) services under battery conditioning constraints, aiming to balance the interests of the DSO, EV aggregators (EVAs), and EV users. It introduces a hierarchical framework that combines Multi-Agent Constrained Policy Optimization (MACPO) with a Proof-of-Stake (PoS) power allocation to ensure safe, scalable coordination while integrating SOC, SOP, and SOH into the decision process. Key contributions include the SOC/SOP/SOH-aware MACPO-based EVA coordination, the PoS-inspired decentralized power allocation, and a rigorous simulation showing improved grid stability, reduced battery aging, and favorable cost outcomes across multiple stakeholders. The results demonstrate the approach’s potential for enabling high-penetration renewable energy integration through robust, large-scale V2G coordination, with practical implications for grid reliability and EV participation incentives.

Abstract

As electric vehicles (EV) become more prevalent and advances in electric vehicle electronics continue, vehicle-to-grid (V2G) techniques and large-scale scheduling strategies are increasingly important to promote renewable energy utilization and enhance the stability of the power grid. This study proposes a hierarchical multistakeholder V2G coordination strategy based on safe multi-agent constrained deep reinforcement learning (MCDRL) and the Proof-of-Stake algorithm to optimize benefits for all stakeholders, including the distribution system operator (DSO), electric vehicle aggregators (EVAs) and EV users. For DSO, the strategy addresses load fluctuations and the integration of renewable energy. For EVAs, energy constraints and charging costs are considered. The three critical parameters of battery conditioning, state of charge (SOC), state of power (SOP), and state of health (SOH), are crucial to the participation of EVs in V2G. Hierarchical multi-stakeholder V2G coordination significantly enhances the integration of renewable energy, mitigates load fluctuations, meets the energy demands of the EVAs, and reduces charging costs and battery degradation simultaneously.

SOC-Boundary and Battery Aging Aware Hierarchical Coordination of Multiple EV Aggregates Among Multi-stakeholders with Multi-Agent Constrained Deep Reinforcement Learning

TL;DR

The paper tackles the challenge of coordinating large-scale EVs for vehicle-to-grid (V2G) services under battery conditioning constraints, aiming to balance the interests of the DSO, EV aggregators (EVAs), and EV users. It introduces a hierarchical framework that combines Multi-Agent Constrained Policy Optimization (MACPO) with a Proof-of-Stake (PoS) power allocation to ensure safe, scalable coordination while integrating SOC, SOP, and SOH into the decision process. Key contributions include the SOC/SOP/SOH-aware MACPO-based EVA coordination, the PoS-inspired decentralized power allocation, and a rigorous simulation showing improved grid stability, reduced battery aging, and favorable cost outcomes across multiple stakeholders. The results demonstrate the approach’s potential for enabling high-penetration renewable energy integration through robust, large-scale V2G coordination, with practical implications for grid reliability and EV participation incentives.

Abstract

As electric vehicles (EV) become more prevalent and advances in electric vehicle electronics continue, vehicle-to-grid (V2G) techniques and large-scale scheduling strategies are increasingly important to promote renewable energy utilization and enhance the stability of the power grid. This study proposes a hierarchical multistakeholder V2G coordination strategy based on safe multi-agent constrained deep reinforcement learning (MCDRL) and the Proof-of-Stake algorithm to optimize benefits for all stakeholders, including the distribution system operator (DSO), electric vehicle aggregators (EVAs) and EV users. For DSO, the strategy addresses load fluctuations and the integration of renewable energy. For EVAs, energy constraints and charging costs are considered. The three critical parameters of battery conditioning, state of charge (SOC), state of power (SOP), and state of health (SOH), are crucial to the participation of EVs in V2G. Hierarchical multi-stakeholder V2G coordination significantly enhances the integration of renewable energy, mitigates load fluctuations, meets the energy demands of the EVAs, and reduces charging costs and battery degradation simultaneously.
Paper Structure (36 sections, 38 equations, 10 figures, 2 tables)

This paper contains 36 sections, 38 equations, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. DSO Coordination Strategy
  4. Battery Aging Aware Coordination
  5. EV Battery Simulation Model
  6. Scope and Contributions
  7. Problem Formulation
  8. MACPO and PoS-based Multi-Stakeholder Hierarchical V2G Coordination Strategy
  9. MACPO-based EVA Coordination
  10. Agent
  11. Environment & State
  12. Action
  13. Reward
  14. Cost
  15. PoS-inspired EVA Power Allocation Algorithm
  16. ...and 21 more sections

Figures (10)

  • Figure 1: The workflow of the proposed multi-stakeholder hierarchical V2G coordination strategy.
  • Figure 2: The MACPO architecture for training and execution.
  • Figure 3: Flowchart of the PoS-inspired EVA power allocation algorithm.
  • Figure 4: Daily load profiles in a microgrid.
  • Figure 5: Electric vehicle battery simulation system.
  • ...and 5 more figures