Profit Maximization for Electric Vehicle Charging Stations Using Multiagent Reinforcement Learning

TL;DR

This work tackles profit maximization for a network of EVCSs each equipped with ESS and RES under uncertainty in EV demand and real-time prices. It introduces a cooperative MARL framework based on Double Hypernetwork QMIX to coordinate energy management and internal trading among EVCSs, mitigating overestimation bias through dual hypernetworks and a min-output mixing strategy. Key contributions include a distributed coordination scheme, an integrated internal energy trading mechanism, and empirical evidence that the approach outperforms standard QMIX and other baselines, achieving profits close to an ideal foreknowledge upper bound and robust performance under demand and renewable fluctuations. The findings underscore the practical potential of advanced MARL for scalable, profit-oriented energy management in multi-EV charging networks with renewable integration.

Abstract

Electric vehicles (EVs) are increasingly integrated into power grids, offering economic and environmental benefits but introducing challenges due to uncoordinated charging. This study addresses the profit maximization problem for multiple EV charging stations (EVCSs) equipped with energy storage systems (ESS) and renewable energy sources (RES), with the capability for energy trading. We propose a Double Hypernetwork QMIX-based multi-agent reinforcement learning (MARL) framework to optimize cooperative energy management under uncertainty in EV demand, renewable generation, and real-time electricity prices. The framework mitigates overestimation bias in value estimation, enables distributed decision-making, and incorporates an internal energy trading mechanism. Numerical experiments using real-world data demonstrate that, compared to standard QMIX, the proposed method achieves approximately 5.3% and 12.7% higher total profit for the two regions, respectively, highlighting its economic and operational efficiency. Additionally, the approach maintains robust performance under varying levels of EV demand uncertainty and renewable energy fluctuations.

Figures (4)

• Figure 1: Overview of EVCS systems and components.
• Figure 2: Implementation of Double Hypernetwork QMIX for EVCS Energy Management.
• Figure 3: Charging or discharging profiles $E_{\text{cs},i}(t)$ at EVCSs in response to renewable energy generation $E_{\text{r},i}(t)$, EV energy demand $D_{\text{ev},i}^{\text{U}}(t)$, $D_{\text{ev},i}^{\text{R}}(t)$, and electricity price $\xi_{\text{u}}(t)$.
• Figure 4: Learning curves of the proposed method and comparable algorithms, showing the average monthly profit across five random seeds. The proposed method demonstrates superior performance and stability, closely approaching the theoretical optimum.