Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integration of Multi-Mode Preference into Home Energy Management System Using Deep Reinforcement Learning

Mohammed Sumayli, Olugbenga Moses Anubi

TL;DR

The paper tackles dynamic consumer comfort within Home Energy Management Systems (HEMS) for Demand Response (DR) programs, arguing that static, fixed comfort weights fail to capture evolving household preferences. It proposes a dynamic, multi-mode DRL-HEMS using a model-free, single-agent deep reinforcement learning approach, specifically a Duelling Double DQN, to adapt per-appliance preferences in real time. The framework defines three DR modes (baseline, moderate, and high flexibility) and models time-shiftable loads, controllable HVAC, and EV charging under day-ahead price signals, evaluated on real-world data to demonstrate near-optimal energy costs with significantly faster computation than MILP. Results show the DRL-HEMS achieving costs close to MILP (approximately a 2.5% gap) while delivering real-time decisions in around 1 second, indicating strong practical potential for scalable, adaptive, and user-centered DR in residential settings.

Abstract

Home Energy Management Systems (HEMS) have emerged as a pivotal tool in the smart home ecosystem, aiming to enhance energy efficiency, reduce costs, and improve user comfort. By enabling intelligent control and optimization of household energy consumption, HEMS plays a significant role in bridging the gap between consumer needs and energy utility objectives. However, much of the existing literature construes consumer comfort as a mere deviation from the standard appliance settings. Such deviations are typically incorporated into optimization objectives via static weighting factors. These factors often overlook the dynamic nature of consumer behaviors and preferences. Addressing this oversight, our paper introduces a multi-mode Deep Reinforcement Learning-based HEMS (DRL-HEMS) framework, meticulously designed to optimize based on dynamic, consumer-defined preferences. Our primary goal is to augment consumer involvement in Demand Response (DR) programs by embedding dynamic multi-mode preferences tailored to individual appliances. In this study, we leverage a model-free, single-agent DRL algorithm to deliver a HEMS framework that is not only dynamic but also user-friendly. To validate its efficacy, we employed real-world data at 15-minute intervals, including metrics such as electricity price, ambient temperature, and appliances' power consumption. Our results show that the model performs exceptionally well in optimizing energy consumption within different preference modes. Furthermore, when compared to traditional algorithms based on Mixed-Integer Linear Programming (MILP), our model achieves nearly optimal performance while outperforming in computational efficiency.

Integration of Multi-Mode Preference into Home Energy Management System Using Deep Reinforcement Learning

TL;DR

The paper tackles dynamic consumer comfort within Home Energy Management Systems (HEMS) for Demand Response (DR) programs, arguing that static, fixed comfort weights fail to capture evolving household preferences. It proposes a dynamic, multi-mode DRL-HEMS using a model-free, single-agent deep reinforcement learning approach, specifically a Duelling Double DQN, to adapt per-appliance preferences in real time. The framework defines three DR modes (baseline, moderate, and high flexibility) and models time-shiftable loads, controllable HVAC, and EV charging under day-ahead price signals, evaluated on real-world data to demonstrate near-optimal energy costs with significantly faster computation than MILP. Results show the DRL-HEMS achieving costs close to MILP (approximately a 2.5% gap) while delivering real-time decisions in around 1 second, indicating strong practical potential for scalable, adaptive, and user-centered DR in residential settings.

Abstract

Home Energy Management Systems (HEMS) have emerged as a pivotal tool in the smart home ecosystem, aiming to enhance energy efficiency, reduce costs, and improve user comfort. By enabling intelligent control and optimization of household energy consumption, HEMS plays a significant role in bridging the gap between consumer needs and energy utility objectives. However, much of the existing literature construes consumer comfort as a mere deviation from the standard appliance settings. Such deviations are typically incorporated into optimization objectives via static weighting factors. These factors often overlook the dynamic nature of consumer behaviors and preferences. Addressing this oversight, our paper introduces a multi-mode Deep Reinforcement Learning-based HEMS (DRL-HEMS) framework, meticulously designed to optimize based on dynamic, consumer-defined preferences. Our primary goal is to augment consumer involvement in Demand Response (DR) programs by embedding dynamic multi-mode preferences tailored to individual appliances. In this study, we leverage a model-free, single-agent DRL algorithm to deliver a HEMS framework that is not only dynamic but also user-friendly. To validate its efficacy, we employed real-world data at 15-minute intervals, including metrics such as electricity price, ambient temperature, and appliances' power consumption. Our results show that the model performs exceptionally well in optimizing energy consumption within different preference modes. Furthermore, when compared to traditional algorithms based on Mixed-Integer Linear Programming (MILP), our model achieves nearly optimal performance while outperforming in computational efficiency.
Paper Structure (18 sections, 25 equations, 8 figures, 4 tables, 1 algorithm)

This paper contains 18 sections, 25 equations, 8 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Overview of RL
  3. Preliminary
  4. Deep Q Networks
  5. HEMS Framework and Modeling of Home Appliances
  6. HEMS Framework
  7. Modeling of Home Appliances and EV
  8. Time-Shiftable Appliances.
  9. Controllable Appliances
  10. Electric Vehicle
  11. Methodology
  12. MDP Formulation
  13. State
  14. Action
  15. Reward
  16. ...and 3 more sections

Figures (8)

  • Figure 1: Framework of multi-mode HEMS. Icons sourced from Flaticon (https://www.flaticon.com).
  • Figure 2: Structural diagram of the Dueling Double DQN algorithm. Icons sourced from Flaticon (https://www.flaticon.com).
  • Figure 3: Rewards over the number of training episodes
  • Figure 4: Scheduling of dishwasher with different preferences
  • Figure 5: Scheduling of washing machine with different preferences
  • ...and 3 more figures