FRESCO: Federated Reinforcement Energy System for Cooperative Optimization

TL;DR

The paper addresses coordinating distributed microgrid resources under privacy constraints to curb carbon emissions. It introduces FRESCO, a three-layer hierarchical RL framework trained with federated learning (FedAVG) in an OpenAI Gym environment, complemented by a synthetic data generator; key gains are quantified with $P_{\Delta}=P_{base}-P_{FRESCO}$ and $C_{\Delta}=C_{base}-C_{FRESCO}$. It demonstrates that lower-layer greedy agents can be aligned by higher-layer controllers to achieve cooperative objectives, while FRL accelerates policy convergence with only a modest impact on training speed. The work suggests that privacy-preserving, scalable energy markets across multiple microgrids can reduce bills and CO2 emissions while enhancing market participation.

Abstract

The rise in renewable energy is creating new dynamics in the energy grid that promise to create a cleaner and more participative energy grid, where technology plays a crucial part in making the required flexibility to achieve the vision of the next-generation grid. This work presents FRESCO, a framework that aims to ease the implementation of energy markets using a hierarchical control architecture of reinforcement learning agents trained using federated learning. The core concept we are proving is that having greedy agents subject to changing conditions from a higher level agent creates a cooperative setup that will allow for fulfilling all the individual objectives. This paper presents a general overview of the framework, the current progress, and some insights we obtained from the recent results.

Figures (3)

• Figure 1: Three-layer architecture of our hierarchical microgrid control
• Figure 2: Reinforcement learning A2C run
• Figure 3: Federated Learning Run