Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Human-in-the-Loop AI for HVAC Management Enhancing Comfort and Energy Efficiency

Xinyu Liang, Frits de Nijs, Buser Say, Hao Wang

TL;DR

This work tackles the high energy footprint of HVAC systems by introducing a human-in-the-loop reinforcement learning framework that integrates real-time occupant feedback and wholesale electricity price signals into an MDP for HVAC control. It combines a two-layer bidirectional LSTM occupancy predictor with PPO-based policy optimization, enabling the system to learn comfort preferences on the fly while exploiting dynamic energy prices for demand response. The approach eliminates reliance on predefined comfort models, achieving significant cost reductions while maintaining or improving occupant comfort, and approaching the performance of an ideal, perfect-prediction optimizer under realistic conditions. The method promises scalable, privacy-conscious, and grid-responsive HVAC management suitable for diverse building types and grid conditions.

Abstract

Heating, Ventilation, and Air Conditioning (HVAC) systems account for approximately 38% of building energy consumption globally, making them one of the most energy-intensive services. The increasing emphasis on energy efficiency and sustainability, combined with the need for enhanced occupant comfort, presents a significant challenge for traditional HVAC systems. These systems often fail to dynamically adjust to real-time changes in electricity market rates or individual comfort preferences, leading to increased energy costs and reduced comfort. In response, we propose a Human-in-the-Loop (HITL) Artificial Intelligence framework that optimizes HVAC performance by incorporating real-time user feedback and responding to fluctuating electricity prices. Unlike conventional systems that require predefined information about occupancy or comfort levels, our approach learns and adapts based on ongoing user input. By integrating the occupancy prediction model with reinforcement learning, the system improves operational efficiency and reduces energy costs in line with electricity market dynamics, thereby contributing to demand response initiatives. Through simulations, we demonstrate that our method achieves significant cost reductions compared to baseline approaches while maintaining or enhancing occupant comfort. This feedback-driven approach ensures personalized comfort control without the need for predefined settings, offering a scalable solution that balances individual preferences with economic and environmental goals.

Human-in-the-Loop AI for HVAC Management Enhancing Comfort and Energy Efficiency

TL;DR

This work tackles the high energy footprint of HVAC systems by introducing a human-in-the-loop reinforcement learning framework that integrates real-time occupant feedback and wholesale electricity price signals into an MDP for HVAC control. It combines a two-layer bidirectional LSTM occupancy predictor with PPO-based policy optimization, enabling the system to learn comfort preferences on the fly while exploiting dynamic energy prices for demand response. The approach eliminates reliance on predefined comfort models, achieving significant cost reductions while maintaining or improving occupant comfort, and approaching the performance of an ideal, perfect-prediction optimizer under realistic conditions. The method promises scalable, privacy-conscious, and grid-responsive HVAC management suitable for diverse building types and grid conditions.

Abstract

Heating, Ventilation, and Air Conditioning (HVAC) systems account for approximately 38% of building energy consumption globally, making them one of the most energy-intensive services. The increasing emphasis on energy efficiency and sustainability, combined with the need for enhanced occupant comfort, presents a significant challenge for traditional HVAC systems. These systems often fail to dynamically adjust to real-time changes in electricity market rates or individual comfort preferences, leading to increased energy costs and reduced comfort. In response, we propose a Human-in-the-Loop (HITL) Artificial Intelligence framework that optimizes HVAC performance by incorporating real-time user feedback and responding to fluctuating electricity prices. Unlike conventional systems that require predefined information about occupancy or comfort levels, our approach learns and adapts based on ongoing user input. By integrating the occupancy prediction model with reinforcement learning, the system improves operational efficiency and reduces energy costs in line with electricity market dynamics, thereby contributing to demand response initiatives. Through simulations, we demonstrate that our method achieves significant cost reductions compared to baseline approaches while maintaining or enhancing occupant comfort. This feedback-driven approach ensures personalized comfort control without the need for predefined settings, offering a scalable solution that balances individual preferences with economic and environmental goals.
Paper Structure (47 sections, 34 equations, 3 figures, 1 table)

This paper contains 47 sections, 34 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Model and Formulation
  3. MDP Model of HITL HVAC Control Problem
  4. State Space Representation
  5. Indoor Temperature ($T^\text{in}_{t}$)
  6. Outdoor Temperature ($T^\text{out}_{t}$)
  7. Time of Day ($\tau$)
  8. Forecasted Outdoor Temperatures ($\tilde{\textbf{T}}^\text{out}_{t}$)
  9. Occupancy Information ($O_{t}$)
  10. Forecasted Occupancy Information ($\tilde{\mathbf{O}}_{t}$)
  11. Feedback Buffer ($\mathbf{f}_t$)
  12. Wholesale Market Rate ($\rho_{t}$)
  13. Forecasted Wholesale Market Rate ($\tilde{\boldsymbol{\rho}}_{t}$)
  14. Action Space Definition
  15. State Transition Dynamics
  16. ...and 32 more sections

Figures (3)

  • Figure 1: Overview of HITL Energy Management System.
  • Figure 2: Comparison of different control strategies in terms of objectives, temperatures, and control decisions plus feedback.
  • Figure 3: Sensitivity analysis of the HITL framework under different maximum probability caps $p^\text{max}$ with discomfort proportion $\beta$ = 0.5.