Multiagent Reinforcement Learning in Enhancing Resilience of Microgrids under Extreme Weather Events

TL;DR

This paper tackles the challenge of ensuring microgrid resilience during extreme weather, where load and renewable generation are highly uncertain. It introduces a cooperative multi-agent deep reinforcement learning framework (MADRL) that employs GRU-based temporal feature extraction and action masking to coordinate distributed energy resources across multiple ESS agents in islanded operation. The approach, rooted in a MADDPG-like actor-critic structure, demonstrates superior resilience and lower operating costs on the IEEE 33-bus system using real PV/load data, and shows robustness to forecast errors and partial agent failures relative to baselines such as MINLP, DDPG, and MAPPO. The work has practical significance for scalable, real-time EMS in microgrids facing typhoon- and other extreme-weather scenarios, with potential extensions to multi-microgrid configurations and cross-topology generalization.

Abstract

Grid resilience is crucial in light of power interruptions caused by increasingly frequent extreme weather events. Well-designed energy management systems (EMS) have made progress in improving microgrid resilience through the coordination of distributed energy resources (DERs), but still face significant challenges in addressing the uncertainty of load demand caused by extreme weather. The integration of deep reinforcement learning (DRL) into EMS design enables optimized microgrid control strategies for coordinating DERs. Building on this, we proposed a cooperative multi-agent deep reinforcement learning (MADRL)-based EMS framework to provide flexible scalability for microgrids, enhance resilience and reduce operational costs during power outages. Specifically, the gated recurrent unit with a gating mechanism was introduced to extract features from temporal data, which enables the EMS to coordinate DERs more efficiently. Next, the proposed MADRL method incorporating action masking techniques was evaluated in the IEEE 33-Bus system using real-world data on renewable generation and power load. Finally, the numerical results demonstrated the superiority of the proposed method in reducing operating costs as well as the effectiveness in enhancing microgrid resilience during power interruptions.

Figures (9)

• Figure 1: Proposed MADRL structure. The structure consists of multiple ESS actor agents and corresponding ESS critic agents. Each ESS actor agent contains an ESS GRU and a SoC Count component, processing state (s) and action (a) information.
• Figure 2: Actor-Critic reinforcement learning structure. It uses an Actor to choose actions based on states, evaluated by a Critic using rewards from the Environment. The Critic's feedback guides updates to both the Actor's policy and the Critic's value function for continual performance improvement.
• Figure 3: Action masking workflow for accelerating reinforcement learning convergence.
• Figure 4: Mapping the neural network output into real ESS power.
• Figure 5: IEEE 33-bus Distribution System with Integrated Distributed Energy Resources (DERs): A Schematic Representation of PV, ESS, and Generator Integration