Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safe Bottom-Up Flexibility Provision from Distributed Energy Resources

Costas Mylonas, Emmanouel Varvarigos, Georgios Tsaousoglou

TL;DR

The paper tackles the challenge of enabling bottom-up flexibility from distributed energy resources while guaranteeing distribution-network safety without active involvement from DSOs. It develops Safe MADDPG, a decentralized multi-agent reinforcement learning framework that uses a model-free voltage predictor in a safety layer to enforce voltage bounds, enabling private prosumers to participate in real-time flexibility markets. Key contributions include decentralized decision-making with intertemporal energy and DR constraints, hard voltage safety via a data-driven projector, and a rigorous evaluation on the IEEE 33-bus network showing zero voltage violations and competitive performance relative to an optimal baseline. The approach advances practical deployment of DER-driven flexibility by preserving owner control, scalability, and grid safety in data-driven, topology-agnostic settings.

Abstract

Modern renewables-based power systems need to tap on the flexibility of Distributed Energy Resources (DERs) connected to distribution networks. It is important, however, that DER owners/users remain in control of their assets, decisions, and objectives. At the same time, the dynamic landscape of DER-penetrated distribution networks calls for agile, data-driven flexibility management frameworks. In the face of these developments, the Multi-Agent Reinforcement Learning (MARL) paradigm is gaining significant attention, as a distributed and data-driven decision-making policy. This paper addresses the need for bottom-up DER management decisions to account for the distribution network's safety-related constraints. While the related literature on safe MARL typically assumes that network characteristics are available and incorporated into the policy's safety layer, which implies active DSO engagement, this paper ensures that self-organized DER communities are enabled to provide distribution-network-safe flexibility services without relying on the aspirational and problematic requirement of bringing the DSO in the decision-making loop.

Safe Bottom-Up Flexibility Provision from Distributed Energy Resources

TL;DR

The paper tackles the challenge of enabling bottom-up flexibility from distributed energy resources while guaranteeing distribution-network safety without active involvement from DSOs. It develops Safe MADDPG, a decentralized multi-agent reinforcement learning framework that uses a model-free voltage predictor in a safety layer to enforce voltage bounds, enabling private prosumers to participate in real-time flexibility markets. Key contributions include decentralized decision-making with intertemporal energy and DR constraints, hard voltage safety via a data-driven projector, and a rigorous evaluation on the IEEE 33-bus network showing zero voltage violations and competitive performance relative to an optimal baseline. The approach advances practical deployment of DER-driven flexibility by preserving owner control, scalability, and grid safety in data-driven, topology-agnostic settings.

Abstract

Modern renewables-based power systems need to tap on the flexibility of Distributed Energy Resources (DERs) connected to distribution networks. It is important, however, that DER owners/users remain in control of their assets, decisions, and objectives. At the same time, the dynamic landscape of DER-penetrated distribution networks calls for agile, data-driven flexibility management frameworks. In the face of these developments, the Multi-Agent Reinforcement Learning (MARL) paradigm is gaining significant attention, as a distributed and data-driven decision-making policy. This paper addresses the need for bottom-up DER management decisions to account for the distribution network's safety-related constraints. While the related literature on safe MARL typically assumes that network characteristics are available and incorporated into the policy's safety layer, which implies active DSO engagement, this paper ensures that self-organized DER communities are enabled to provide distribution-network-safe flexibility services without relying on the aspirational and problematic requirement of bringing the DSO in the decision-making loop.
Paper Structure (19 sections, 15 equations, 5 figures, 1 table, 1 algorithm)

This paper contains 19 sections, 15 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation
  3. Related Work
  4. Research Gap and Contributions
  5. System Model
  6. DER Models
  7. Distribution Network Model
  8. Problem Formulation
  9. Proposed Solution
  10. Multi-Agent Deep Reinforcement Learning
  11. Safety Layer
  12. The Algorithm
  13. Case Study - Results
  14. Case Study Description
  15. Voltage Regressor
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Illustration of the safety layer used in combination with the MADDPG networks.
  • Figure 2: Predicted and actual voltage levels for bus 5 over 24 hours.
  • Figure 3: (a) Episode reward over training (left) and (b) normalized voltage-violation cost over training (right).
  • Figure 4: Comparison of power reduction per building between the OPF solution and the Safe-MADDPG policy for a single day, alongside flexibility price and buy prices.
  • Figure 5: Comparison of ESS energy levels per building between the OPF solution and the Safe-MADDPG policy for a single day, alongside flexibility and buy prices.