Toward Adaptive Grid Resilience: A Gradient-Free Meta-RL Framework for Critical Load Restoration

TL;DR

This paper tackles critical-load restoration in renewable-rich distribution grids under uncertainty by introducing MGF-RL, a gradient-free, meta-guided reinforcement learning framework. It combines ES-based within-task optimization with a first-order meta-update to learn a transferable initialization, enabling rapid adaptation to unseen outage scenarios while avoiding second-order derivatives. The approach is validated on IEEE-13 and IEEE-123 bus systems, showing substantial improvements in reliability metrics (e.g., SAIDI) and earlier restoration, especially under forecast errors, and is supported by theoretical regret guarantees in static and dynamic environments. The work offers a practical and scalable alternative to model-based optimization, with strong potential for real-time deployment and future enhancements including dynamic topology and hybrid robust-optimization strategies.

Abstract

Restoring critical loads after extreme events demands adaptive control to maintain distribution-grid resilience, yet uncertainty in renewable generation, limited dispatchable resources, and nonlinear dynamics make effective restoration difficult. Reinforcement learning (RL) can optimize sequential decisions under uncertainty, but standard RL often generalizes poorly and requires extensive retraining for new outage configurations or generation patterns. We propose a meta-guided gradient-free RL (MGF-RL) framework that learns a transferable initialization from historical outage experiences and rapidly adapts to unseen scenarios with minimal task-specific tuning. MGF-RL couples first-order meta-learning with evolutionary strategies, enabling scalable policy search without gradient computation while accommodating nonlinear, constrained distribution-system dynamics. Experiments on IEEE 13-bus and IEEE 123-bus test systems show that MGF-RL outperforms standard RL, MAML-based meta-RL, and model predictive control across reliability, restoration speed, and adaptation efficiency under renewable forecast errors. MGF-RL generalizes to unseen outages and renewable patterns while requiring substantially fewer fine-tuning episodes than conventional RL. We also provide sublinear regret bounds that relate adaptation efficiency to task similarity and environmental variation, supporting the empirical gains and motivating MGF-RL for real-time load restoration in renewable-rich distribution grids.

Figures (9)

• Figure 1: The CLR learning environment integrates a DRL agent with OpenDSS for optimizing load restoration through iterative policy improvement.
• Figure 2: Cumulative reward curves for RL algorithms optimizing $f_{1}(x)=-x^{2}+10$ and 2D Ackley function $f_{2}(x,y)=100-20\exp(-0.2\sqrt{0.5(x^{2}+y^{2})})-\exp(0.5(cos(2\pi x)+cos(2\pi y )))+e+20$. The global search capability of ES-RL enables faster convergence and higher final rewards.
• Figure 3: Modified IEEE-13 bus system
• Figure 4: Active power demand profiles (kW) across 60 restoration tasks. The x-axis represents Task ID, the y-axis shows load identifiers (node and phase, e.g., '675c' indicates node 675, phase c), and the z-axis indicates active power demand. Loads without phase labels are balanced three-phase loads. The substantial variation demonstrates task diversity for meta-learning evaluation.
• Figure 5: Learning curves showing mean and variance of episode rewards over 5 runs, with MGF-RL achieving higher and stable performance across tasks.

Theorems & Definitions (23)

• Theorem 4.1: Theorem 6; analysis
• Theorem 4.2: TAOG for static environment
• Remark 1
• Theorem 4.3: Dynamic regret for online learning
• Theorem 4.4: Regret with adaptive learning rate
• Remark 2
• Theorem 4.5: TAOG for dynamic environment
• Remark 3
• Theorem C.1
• Proof C.1