Towards Efficient Multi-Objective Optimisation for Real-World Power Grid Topology Control

TL;DR

This paper tackles the challenge of real-world power-grid topology control under multi-objective trade-offs, where traditional MORL approaches struggle with large state and action spaces. It introduces a two-phase approach: Phase 1 trains a reinforcement learning agent to optimize a utility reward that balances $\max\rho_{n-1}$ and topology-change costs, and Phase 2 applies Single-Step Planning (SSP) to generate a non-dominated set of day-ahead plans that approximate the Pareto Front. The authors implement two RL variants—Single-Step Agent (SSA) and AlphaZero Agent (AZA)—and combine them with SSP to produce multiple feasible plans per day, validated on real-world TenneT data; results show strong hypervolume performance, lower worst-case N-1 load, and high plan solvability with practical computation times (~4–7 minutes). The work demonstrates a scalable, computationally efficient framework that outperforms fixed expert baselines and offers concrete operational benefits for TSOs, with clear avenues for extending to additional objectives and plan diversity.

Abstract

Power grid operators face increasing difficulties in the control room as the increase in energy demand and the shift to renewable energy introduce new complexities in managing congestion and maintaining a stable supply. Effective grid topology control requires advanced tools capable of handling multi-objective trade-offs. While Reinforcement Learning (RL) offers a promising framework for tackling such challenges, existing Multi-Objective Reinforcement Learning (MORL) approaches fail to scale to the large state and action spaces inherent in real-world grid operations. Here we present a two-phase, efficient and scalable Multi-Objective Optimisation (MOO) method designed for grid topology control, combining an efficient RL learning phase with a rapid planning phase to generate day-ahead plans for unseen scenarios. We validate our approach using historical data from TenneT, a European Transmission System Operator (TSO), demonstrating minimal deployment time, generating day-ahead plans within 4-7 minutes with strong performance. These results underline the potential of our scalable method to support real-world power grid management, offering a practical, computationally efficient, and time-effective tool for operational planning. Based on current congestion costs and inefficiencies in grid operations, adopting our approach by TSOs could potentially save millions of euros annually, providing a compelling economic incentive for its integration in the control room.

Figures (4)

• Figure 1: Distribution of hypervolume values by approach and baseline across the three data splits. Higher values indicate better performance.
• Figure 2: Statistics considering the best plan that yields the best $\max\rho_{n-1}$.
• Figure 3: Depiction of part of the grid used in our experiments. The entire grid contains 1659 substations, 1338 power lines and 1716 injections.
• Figure 4: Corresponding "N switching" objective average for best plans with best $\max\rho_{n-1}$ depicted in Fig.\ref{['fig:best_max_n_1_and_n_solved_days']}.