Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Applications in CityLearn Gym Environment for Multi-Objective Control Benchmarking in Grid-Interactive Buildings and Districts

Kingsley Nweye, Zoltan Nagy

TL;DR

The paper addresses the challenge of benchmarking multi-resource control in grid-interactive buildings and districts by leveraging CityLearn v2, an open-source Gym environment. It systematically compares baselines, rule-based controls, and reinforcement learning controllers across 17 city-scale configurations using a two-building Austin dataset with heat pumps, storage, and PV, assessing objectives including cost, emissions, comfort, and peak demand. Key findings show that while sophisticated control can yield notable cost reductions, emissions reductions are not consistently achieved and gains depend on DER sizing and reward design; RL can reduce discomfort and consumption with carefully tuned multi-objective rewards, and district-peak reductions are modest but attainable with appropriate control. Overall, the work demonstrates CityLearn as a practical, extensible benchmark for algorithm benchmarking in grid-interactive buildings and districts, and highlights directions for improving signal quality, expert-guided training, and scaling to more diverse stock and objectives.

Abstract

It is challenging to coordinate multiple distributed energy resources in a single or multiple buildings to ensure efficient and flexible operation. Advanced control algorithms such as model predictive control and reinforcement learning control provide solutions to this problem by effectively managing a distribution of distributed energy resource control tasks while adapting to unique building characteristics, and cooperating towards improving multi-objective key performance indicator. Yet, a research gap for advanced control adoption is the ability to benchmark algorithm performance. CityLearn addresses this gap an open-source Gym environment for the easy implementation and benchmarking of simple rule-based control and advanced algorithms that has an advantage of modeling simplicity, multi-agent control, district-level objectives, and control resiliency assessment. Here we demonstrate the functionalities of CityLearn using 17 different building control problems that have varying complexity with respect to the number of controllable distributed energy resources in buildings, the simplicity of the control algorithm, the control objective, and district size.

Applications in CityLearn Gym Environment for Multi-Objective Control Benchmarking in Grid-Interactive Buildings and Districts

TL;DR

The paper addresses the challenge of benchmarking multi-resource control in grid-interactive buildings and districts by leveraging CityLearn v2, an open-source Gym environment. It systematically compares baselines, rule-based controls, and reinforcement learning controllers across 17 city-scale configurations using a two-building Austin dataset with heat pumps, storage, and PV, assessing objectives including cost, emissions, comfort, and peak demand. Key findings show that while sophisticated control can yield notable cost reductions, emissions reductions are not consistently achieved and gains depend on DER sizing and reward design; RL can reduce discomfort and consumption with carefully tuned multi-objective rewards, and district-peak reductions are modest but attainable with appropriate control. Overall, the work demonstrates CityLearn as a practical, extensible benchmark for algorithm benchmarking in grid-interactive buildings and districts, and highlights directions for improving signal quality, expert-guided training, and scaling to more diverse stock and objectives.

Abstract

It is challenging to coordinate multiple distributed energy resources in a single or multiple buildings to ensure efficient and flexible operation. Advanced control algorithms such as model predictive control and reinforcement learning control provide solutions to this problem by effectively managing a distribution of distributed energy resource control tasks while adapting to unique building characteristics, and cooperating towards improving multi-objective key performance indicator. Yet, a research gap for advanced control adoption is the ability to benchmark algorithm performance. CityLearn addresses this gap an open-source Gym environment for the easy implementation and benchmarking of simple rule-based control and advanced algorithms that has an advantage of modeling simplicity, multi-agent control, district-level objectives, and control resiliency assessment. Here we demonstrate the functionalities of CityLearn using 17 different building control problems that have varying complexity with respect to the number of controllable distributed energy resources in buildings, the simplicity of the control algorithm, the control objective, and district size.
Paper Structure (15 sections, 4 figures, 6 tables)

This paper contains 15 sections, 4 figures, 6 tables.

Table of Contents

  1. Abstract
  2. Introduction
  3. CityLearn
  4. Application
  5. Dataset
  6. Control Simulations
  7. Baseline (No Control)
  8. Rule-Based Control
  9. Reinforcement Learning Control
  10. Results
  11. Single Building Cost or Emissions Objective
  12. Single Building Discomfort and Consumption Objective
  13. District Average Daily Peak Objective
  14. Discussion
  15. Conclusion

Figures (4)

  • Figure 1: CityLearn environment and control interaction.
  • Figure 2: Cost ($) and emissions (kgCO2e) from either or of , - system or both when the control objective is cost or emission. The dashed red line shows the cost for a no-control scenario and the dotted blue line shows the cost for a no-control scenario but with solar generation to augment electricity from the grid.
  • Figure 3: action and consequent trend in the initial seven days of the two-week evaluation period when the control objective is to either minimize cost or emissions.
  • Figure 4: District-level daily peak load for two-building environments where each building has - system.