Explainable Reinforcement Learning-based Home Energy Management Systems using Differentiable Decision Trees

TL;DR

This work proposes a reinforcement learning-based approach that integrates the scalability of data-driven reinforcement learning with the explainability of differentiable decision trees, which leads to a controller that can be easily adapted across different houses and provides a simple control policy that can be explained to end-users, further improving user acceptance.

Abstract

With the ongoing energy transition, demand-side flexibility has become an important aspect of the modern power grid for providing grid support and allowing further integration of sustainable energy sources. Besides traditional sources, the residential sector is another major and largely untapped source of flexibility, driven by the increased adoption of solar PV, home batteries, and EVs. However, unlocking this residential flexibility is challenging as it requires a control framework that can effectively manage household energy consumption, and maintain user comfort while being readily scalable across different, diverse houses. We aim to address this challenging problem and introduce a reinforcement learning-based approach using differentiable decision trees. This approach integrates the scalability of data-driven reinforcement learning with the explainability of (differentiable) decision trees. This leads to a controller that can be easily adapted across different houses and provides a simple control policy that can be explained to end-users, further improving user acceptance. As a proof-of-concept, we analyze our method using a home energy management problem, comparing its performance with commercially available rule-based baseline and standard neural network-based RL controllers. Through this preliminary study, we show that the performance of our proposed method is comparable to standard RL-based controllers, outperforming baseline controllers by ~20% in terms of daily cost savings while being straightforward to explain.

Figures (6)

• Figure 1: Comparison of DDT-based agents
• Figure 1: Example of a learned DDT for depth 2 showing selected features, learned thresholds and output distributions. The annotations indicate how the DDT can be explained.
• Figure 2: Illustration of a DDT of depth 2 with decision nodes denoted by rounded boxes and leaf nodes with rectangles. Here, all $p_{i}$ represent path probabilities and $p^{L}_{jk}$ represents probabilities at each leaf ($j$) for each element ($k$).
• Figure 3: Daily costs attained by DDT-based agents on the HEMS problem. The dots represent the actual performance of individual models and the box plots show the aggregate performance. DDT-based agents are benchmarked using a standard neural network-based DDPG and a baseline RBC.
• Figure 4: Example of a learned DDT for depth 2 with a redundant decision node