Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fair Reinforcement Learning Algorithm for PV Active Control in LV Distribution Networks

Maurizio Vassallo, Amina Benzerga, Alireza Bahmanyar, Damien Ernst

TL;DR

This work tackles voltage violations in LV distribution networks driven by substantial PV adoption and fairness concerns over active power curtailment. It introduces a deep reinforcement learning approach, specifically a Deep Deterministic Policy Gradient (DDPG) controller, to jointly regulate PV smart-inverter reactive power and, when needed, curtail active power, guided by a reward that combines voltage violation penalties, action magnitude costs, and a fairness term based on curtailment equality. The results across four scenarios show the controller can restore voltages while varying the balance between overall curtailment and fairness, with scenario (d) achieving fairer curtailment distribution at the cost of higher total curtailment compared to some alternatives. The approach offers a scalable path toward fair, voltage-supporting DER integration in distribution networks and suggests directions for extension to larger systems, tuned objective weights, and decentralized reinforcement learning.

Abstract

The increasing adoption of distributed energy resources, particularly photovoltaic (PV) panels, has presented new and complex challenges for power network control. With the significant energy production from PV panels, voltage issues in the network have become a problem. Currently, PV smart inverters (SIs) are used to mitigate the voltage problems by controlling their active power generation and reactive power injection or absorption. However, reducing the active power output of PV panels can be perceived as unfair to some customers, discouraging future installations. To solve this issue, in this paper, a reinforcement learning technique is proposed to address voltage issues in a distribution network, while considering fairness in active power curtailment among customers. The feasibility of the proposed approach is explored through experiments, demonstrating its ability to effectively control voltage in a fair and efficient manner.

Fair Reinforcement Learning Algorithm for PV Active Control in LV Distribution Networks

TL;DR

This work tackles voltage violations in LV distribution networks driven by substantial PV adoption and fairness concerns over active power curtailment. It introduces a deep reinforcement learning approach, specifically a Deep Deterministic Policy Gradient (DDPG) controller, to jointly regulate PV smart-inverter reactive power and, when needed, curtail active power, guided by a reward that combines voltage violation penalties, action magnitude costs, and a fairness term based on curtailment equality. The results across four scenarios show the controller can restore voltages while varying the balance between overall curtailment and fairness, with scenario (d) achieving fairer curtailment distribution at the cost of higher total curtailment compared to some alternatives. The approach offers a scalable path toward fair, voltage-supporting DER integration in distribution networks and suggests directions for extension to larger systems, tuned objective weights, and decentralized reinforcement learning.

Abstract

The increasing adoption of distributed energy resources, particularly photovoltaic (PV) panels, has presented new and complex challenges for power network control. With the significant energy production from PV panels, voltage issues in the network have become a problem. Currently, PV smart inverters (SIs) are used to mitigate the voltage problems by controlling their active power generation and reactive power injection or absorption. However, reducing the active power output of PV panels can be perceived as unfair to some customers, discouraging future installations. To solve this issue, in this paper, a reinforcement learning technique is proposed to address voltage issues in a distribution network, while considering fairness in active power curtailment among customers. The feasibility of the proposed approach is explored through experiments, demonstrating its ability to effectively control voltage in a fair and efficient manner.
Paper Structure (11 sections, 4 equations, 5 figures, 1 table)

This paper contains 11 sections, 4 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Contextual background
  4. Fairness
  5. Smart inverters
  6. Reinforcement learning
  7. Proposed method
  8. Distribution network
  9. RL elements
  10. Results
  11. Conclusion

Figures (5)

  • Figure 1: SI capability curve lr_csi
  • Figure 2: Action-reward feedback loop of a generic RL framework sutton2018reinforcement
  • Figure 3: Actor-critic architecture in DDPG DRLBVCDMUDN
  • Figure 4: The buses' voltage situation for each scenario
  • Figure 5: Gini index for each scenario where some control is applied