Towards Hierarchical Multi-Agent Decision-Making for Uncertainty-Aware EV Charging

TL;DR

HUCA tackles real-time, uncertainty-aware workplace EV charging by introducing a hierarchical multi-agent framework that separates high-level charge/discharge decisions from low-level per-pile power control. A novel uncertainty-aware critic augmentation mechanism enables robust low-level decisions under unpredictable EV departures, while hierarchical coordination reduces building electricity costs. Experimental results on real-world datasets show HUCA achieves lower penalties and competitive total costs compared with baselines, with ablations confirming the value of both hierarchy and the uncertainty augmentation. The work demonstrates practical potential for AI-driven, real-time energy management in office environments.

Abstract

Recent advances in bidirectional EV charging and discharging systems have spurred interest in workplace applications. However, real-world deployments face various dynamic factors, such as fluctuating electricity prices and uncertain EV departure times, that hinder effective energy management. To address these issues and minimize building electricity costs while meeting EV charging requirements, we design a hierarchical multi-agent structure in which a high-level agent coordinates overall charge or discharge decisions based on real-time pricing, while multiple low-level agents manage individual power level accordingly. For uncertain EV departure times, we propose a novel uncertainty-aware critic augmentation mechanism for low-level agents that improves the evaluation of power-level decisions and ensures robust control under such uncertainty. Building upon these two key designs, we introduce HUCA, a real-time charging control framework that coordinates energy supply among the building and EVs. Experiments on real-world electricity datasets show that HUCA significantly reduces electricity costs and maintains competitive performance in meeting EV charging requirements under both simulated certain and uncertain departure scenarios. The results further highlight the importance of hierarchical control and the proposed critic augmentation under the uncertain departure scenario. A case study illustrates HUCA's capability to allocate energy between the building and EVs in real time, underscoring its potential for practical use.

Figures (6)

• Figure 1: The illustration of the workplace charging scenario with the bidirectional EV chargers.
• Figure 2: The overview of the HUCA framework. The high-level agent decides to charge or discharge EVs, while low-level agents control the power of each charging pile within power limitations.
• Figure 4: The illustration of uncertainty-aware critic augmentation. For each low-level agent $c_i$, the action-value function is augmented based on the uncertainty factor (Eq. \ref{['eq:critic_aug']}). This augmentation guides the updated policy to make decisions with explicit consideration of EV departure uncertainty.
• Figure 7: Ablation study on total cost and user satisfaction (CP=20).
• Figure 8: Performance of HUCA with different $\rho$ values with 10 piles in the uncertain departure scenario.