Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Policy Adaptation for Voltage Control Under Unknown Topology Changes

Jie Feng, Yuanyuan Shi, Deepjyoti Deka

TL;DR

The paper tackles voltage control in distribution networks with uncertain topology changes by developing a topology-aware online policy adaptation framework. It combines data-driven estimation of voltage–reactive power sensitivities with a topology-change detector and a sparse line-change identification method, enabling fast updates to a pretrained monotone neural-network controller. The approach yields high line-change identification accuracy with small data windows and substantial improvements in control performance over non-adaptive and regression-based baselines, demonstrated on IEEE 13-bus and SCE 56-bus systems. By exploiting radial structure and change sparsity, it delivers real-time adaptation and robust voltage regulation under topology reconfigurations and load variations, with potential impact on scalable, DER-enabled distribution grids.

Abstract

Reinforcement learning (RL) has shown great potential for designing voltage control policies, but their performance often degrades under changing system conditions such as topology reconfigurations and load variations. We introduce a topology-aware online policy optimization framework that leverages data-driven estimation of voltage-reactive power sensitivities to achieve efficient policy adaptation. Exploiting the sparsity of topology-switching events, where only a few lines change at a time, our method efficiently detects topology changes and identifies the affected lines and parameters, enabling fast and accurate sensitivity updates without recomputing the full sensitivity matrix. The estimated sensitivity is subsequently used for online policy optimization of a pre-trained neural-network-based RL controller. Simulations on both the IEEE 13-bus and SCE 56-bus systems demonstrate over 90 percent line identification accuracy, using only 15 data points. The proposed method also significantly improves voltage regulation performance compared with non-adaptive policies and adaptive policies that rely on regression-based online optimization methods for sensitivity estimation.

Efficient Policy Adaptation for Voltage Control Under Unknown Topology Changes

TL;DR

The paper tackles voltage control in distribution networks with uncertain topology changes by developing a topology-aware online policy adaptation framework. It combines data-driven estimation of voltage–reactive power sensitivities with a topology-change detector and a sparse line-change identification method, enabling fast updates to a pretrained monotone neural-network controller. The approach yields high line-change identification accuracy with small data windows and substantial improvements in control performance over non-adaptive and regression-based baselines, demonstrated on IEEE 13-bus and SCE 56-bus systems. By exploiting radial structure and change sparsity, it delivers real-time adaptation and robust voltage regulation under topology reconfigurations and load variations, with potential impact on scalable, DER-enabled distribution grids.

Abstract

Reinforcement learning (RL) has shown great potential for designing voltage control policies, but their performance often degrades under changing system conditions such as topology reconfigurations and load variations. We introduce a topology-aware online policy optimization framework that leverages data-driven estimation of voltage-reactive power sensitivities to achieve efficient policy adaptation. Exploiting the sparsity of topology-switching events, where only a few lines change at a time, our method efficiently detects topology changes and identifies the affected lines and parameters, enabling fast and accurate sensitivity updates without recomputing the full sensitivity matrix. The estimated sensitivity is subsequently used for online policy optimization of a pre-trained neural-network-based RL controller. Simulations on both the IEEE 13-bus and SCE 56-bus systems demonstrate over 90 percent line identification accuracy, using only 15 data points. The proposed method also significantly improves voltage regulation performance compared with non-adaptive policies and adaptive policies that rely on regression-based online optimization methods for sensitivity estimation.
Paper Structure (28 sections, 1 theorem, 33 equations, 6 figures, 4 tables, 2 algorithms)

This paper contains 28 sections, 1 theorem, 33 equations, 6 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Preliminary
  4. Distribution Grid Modeling
  5. Voltage Control Problem Formulation
  6. Sensitivity Estimation for Data-Driven Voltage Control
  7. Topology Change Detection And Identification
  8. Topology Change Detection
  9. Topology Change Identification
  10. Residual-based topology change modeling
  11. Identification of the involved nodes
  12. Constrained Sparse Identification of Line Changes
  13. Online Policy Optimization for Voltage Control
  14. Online Policy Optimization Objective
  15. Online Gradient-Based Policy Updates
  16. ...and 13 more sections

Key Result

Lemma 3.4

A change event is detected at time $t$ if $\|e_{t}\|_2 >0$. If $\|e_{t+1}\|_2 >0$, the change is due to a topology change, otherwise if $\|e_{t+1}\|_2=0$, the change is attributed to a load change.

Figures (6)

  • Figure 1: Overview of the proposed framework.
  • Figure 2: Diagram of the causal relationships between states, policy parameters, control inputs, and costs when online policy optimization is active. The policy parameters are updated after observing $h_t$ and applied to compute $u_t$.
  • Figure 3: Diagram of a topology change event at time $t$.
  • Figure 4: Diagram for IEEE 13-bus and SCE 56 bus systems.
  • Figure 5: IEEE 13-bus voltage trajectories without control under real-world loads. Left: 24-hour voltage profile with no topology changes. Right: voltage profile when a topology change occurs at 11:40 a.m., disconnecting line (1,5) and connecting line (2,7).
  • ...and 1 more figures

Theorems & Definitions (3)

  • Remark 2.1
  • Lemma 3.4: Detection Criterion under LinDistFlow Model
  • Remark 3.5