An MDP-Based Approach for Distribution System Control with PV Generation and Battery Storage

TL;DR

The paper addresses operational cost minimization in DER-rich distribution networks under PV uncertainty by formulating a discrete-time finite-horizon MDP where the state includes PV prediction error $\delta P_{pv,t}$ and BESS state $E_{bes}$. An offline day-ahead layer computes expected costs and optimal policies via backward induction, while an online layer updates immediate costs with actual data and can extend the action set to hedge forecast errors. Experiments on the IEEE 33-bus system with two DERs show the proposed MDP achieves lower total costs than a deterministic reference, particularly when PV forecast errors are large, and the method also serves as an analytical tool for DER placement. The approach is computationally intensive, motivating future work toward distributed control to improve scalability and real-time applicability, while maintaining the ability to support DSOs in planning and operation.

Abstract

This paper proposes a decision-making approach for the control of distribution systems with distributed energy resources (DERs) equipped with photovoltaic (PV) units and battery energy storage systems (BESS). The objective is to minimize the total operational cost of the distribution system while satisfying the system operating constraints. The method is based on the discrete-time finite-horizon Markov Decision Process (MDP) framework. Different aspects of the distribution system operation are considered, such as the possibilities of curtailment of PV generation, managing battery storage, reactive power injection, load shedding, and providing a flexibility service for the transmission system. The model is tested for the IEEE 33-bus system with two added DERs and the study cases involve various unexpected events. The experimental results show that this method enables the attainment of relatively low total cost values compared to the reference deterministic approach. The benefits of applying this approach are particularly evident when there is a significant difference between the predicted and actual PV power generation.

Figures (8)

• Figure 1: General flow chart of the offline and online calculations.
• Figure 2: Single-line diagram of the IEEE 33-bus distribution system with added DERs.
• Figure 3: Transition probability matrix for the relative error of the PV prediction.
• Figure 4: Load demand profiles (relative to the nominal): predicted, actual, and modified (by adding an unpredicted increase between 10:00 and 14:00).
• Figure 5: PV generation profiles (relative to the maximum) for the first DER: predicted and actual values with small and significant error.