Agent-Based Decentralized Energy Management of EV Charging Station with Solar Photovoltaics via Multi-Agent Reinforcement Learning

TL;DR

The paper tackles robust energy management for EV charging stations with PV in smart cities, addressing uncertainties in demand and charger faults. It introduces a decentralized MARL framework in which each charger is an agent, trained via CTDE with LSTM-based temporal encoding and a dense reward to improve charging satisfaction. The approach optimizes continuous actions $a_{i,t}$ to minimize net costs while accounting for battery degradation via $AGE_{i,t}^{cyc}$, unmet demand $ds_i$, and PV feed-in revenue $a_t^{PVG}$ under constraints such as $G^{max}$ and PV generation $a_t^{PVgen}$. Empirical results on real-world data show robustness to faults and PV variability, with substantial performance gains over baseline MARL methods.

Abstract

In the pursuit of energy net zero within smart cities, transportation electrification plays a pivotal role. The adoption of Electric Vehicles (EVs) keeps increasing, making energy management of EV charging stations critically important. While previous studies have managed to reduce energy cost of EV charging while maintaining grid stability, they often overlook the robustness of EV charging management against uncertainties of various forms, such as varying charging behaviors and possible faults in faults in some chargers. To address the gap, a novel Multi-Agent Reinforcement Learning (MARL) approach is proposed treating each charger to be an agent and coordinate all the agents in the EV charging station with solar photovoltaics in a more realistic scenario, where system faults may occur. A Long Short-Term Memory (LSTM) network is incorporated in the MARL algorithm to extract temporal features from time-series. Additionally, a dense reward mechanism is designed for training the agents in the MARL algorithm to improve EV charging experience. Through validation on a real-world dataset, we show that our approach is robust against system uncertainties and faults and also effective in minimizing EV charging costs and maximizing charging service satisfaction.

Figures (7)

• Figure 1: System model of an EV charging station with solar PV.
• Figure 2: The neural network architecture (left) and the framework of MADDPG (right).
• Figure 3: The agent's action for centralized MARL in the case of partial faults.
• Figure 4: The agent's action for decentralized MARL in the case of partial faults.
• Figure 5: Solar energy under different weather conditions.