Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HVAC-DPT: A Decision Pretrained Transformer for HVAC Control

Anaïs Berkes

TL;DR

This work tackles the scalability and data-efficiency challenges of HVAC control by reframing the problem as a sequential decision task and employing a pretrained transformer with in-context reinforcement learning. The proposed HVAC-DPT framework trains a diverse policy library across multiple buildings, then uses in-context learning to condition a transformer on interaction histories to predict actions without updating model parameters during deployment. In unseen buildings, HVAC-DPT achieves a substantial energy reduction (~$45\%$) compared with a Baseline controller and closely approaches a building-specific Expert, while substantially outperforming RL-based baselines that require extensive retraining. Overall, HVAC-DPT offers scalable generalization for multi-zone HVAC control and reduces the barriers to deploying advanced control strategies across varied building types.

Abstract

Building operations consume approximately 40% of global energy, with Heating, Ventilation, and Air Conditioning (HVAC) systems responsible for up to 50% of this consumption. As HVAC energy demands are expected to rise, optimising system efficiency is crucial for reducing future energy use and mitigating climate change. Existing control strategies lack generalisation and require extensive training and data, limiting their rapid deployment across diverse buildings. This paper introduces HVAC-DPT, a Decision-Pretrained Transformer using in-context Reinforcement Learning (RL) for multi-zone HVAC control. HVAC-DPT frames HVAC control as a sequential prediction task, training a causal transformer on interaction histories generated by diverse RL agents. This approach enables HVAC-DPT to refine its policy in-context, without modifying network parameters, allowing for deployment across different buildings without the need for additional training or data collection. HVAC-DPT reduces energy consumption in unseen buildings by 45% compared to the baseline controller, offering a scalable and effective approach to mitigating the increasing environmental impact of HVAC systems.

HVAC-DPT: A Decision Pretrained Transformer for HVAC Control

TL;DR

This work tackles the scalability and data-efficiency challenges of HVAC control by reframing the problem as a sequential decision task and employing a pretrained transformer with in-context reinforcement learning. The proposed HVAC-DPT framework trains a diverse policy library across multiple buildings, then uses in-context learning to condition a transformer on interaction histories to predict actions without updating model parameters during deployment. In unseen buildings, HVAC-DPT achieves a substantial energy reduction (~) compared with a Baseline controller and closely approaches a building-specific Expert, while substantially outperforming RL-based baselines that require extensive retraining. Overall, HVAC-DPT offers scalable generalization for multi-zone HVAC control and reduces the barriers to deploying advanced control strategies across varied building types.

Abstract

Building operations consume approximately 40% of global energy, with Heating, Ventilation, and Air Conditioning (HVAC) systems responsible for up to 50% of this consumption. As HVAC energy demands are expected to rise, optimising system efficiency is crucial for reducing future energy use and mitigating climate change. Existing control strategies lack generalisation and require extensive training and data, limiting their rapid deployment across diverse buildings. This paper introduces HVAC-DPT, a Decision-Pretrained Transformer using in-context Reinforcement Learning (RL) for multi-zone HVAC control. HVAC-DPT frames HVAC control as a sequential prediction task, training a causal transformer on interaction histories generated by diverse RL agents. This approach enables HVAC-DPT to refine its policy in-context, without modifying network parameters, allowing for deployment across different buildings without the need for additional training or data collection. HVAC-DPT reduces energy consumption in unseen buildings by 45% compared to the baseline controller, offering a scalable and effective approach to mitigating the increasing environmental impact of HVAC systems.

Paper Structure

This paper contains 14 sections, 1 equation, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction and related work
  2. Problem definition
  3. HVAC-DPT
  4. Dataset Generation.
  5. Pretraining.
  6. Online deployment.
  7. Results
  8. Conclusion
  9. Additional system details
  10. Additional experiment details
  11. Dataset Generation
  12. Pretraining
  13. Online Deployment
  14. Additional results

Figures (4)

  • Figure 1: Schematic overview of the proposed methodology.
  • Figure 1: State variables and actions for each agent.
  • Figure 2: HVAC energy consumption (MWh) of different controllers during the first 12 months of deployment in building $B_{Denver}$.
  • Figure 3: Illustration of an air loop in a multi-zone building equipped with a forced-air heating and cooling system gs2023mitigating.