Automation Configuration in Smart Home Systems: Challenges and Opportunities
Sheik Murad Hassan Anik, Xinghua Gao, Hao Zhong, Xiaoyin Wang, Na Meng
TL;DR
This paper addresses the challenges of YAML-based automation configuration in Home Assistant by empirically analyzing 190 HA forum threads (03/2022–08/2022) to characterize user concerns, derive a taxonomy of issues, and evaluate existing YAML tools. It finds that 68% of issues are debugging-oriented, with the Action segment being the most error-prone and 19 distinct technical concepts spanning the debugging set. The authors identify eight recurring resolution strategies and reveal that current tools exhibit high precision but very low recall, detecting only a small fraction of real HA-specific bugs. They open-source their dataset and offer concrete directions for syntactic and semantic tooling as well as data-driven automation generation to improve end-user IoT programming and smart-home reliability.
Abstract
As the innovation of smart devices and internet-of-things (IoT), smart homes have become prevalent. People tend to transform residences into smart homes by customizing off-the-shelf smart home platforms, instead of creating IoT systems from scratch. Among the alternatives, Home Assistant (HA) is one of the most popular platforms. It allows end-users (i.e., home residents) to smartify homes by (S1) integrating selected devices into the system, and (S2) creating YAML files to control those devices. Unfortunately, due to the diversity of devices and complexity of automatic configurations, many users have difficulty correctly creating YAML files. Consequently, their smart homes may not work as expected, causing frustration and concern in users. This paper presents a novel study on issues of YAML-based automation configuration in smart homes (issues related to S2). We mined the online forum Home Assistant Community for discussion threads related to automation configuration. By manually inspecting 190 threads, we revealed 3 categories of concerns: implementation, optimization, and debugging. Under each category, we classified discussions based on the issue locations and technical concepts involved. Among debugging discussions, we further classified discussions based on users' resolution strategies; we also applied existing analysis tools to buggy YAML files, to assess the tool effectiveness. Our study reveals the common challenges faced by users and frequently applied resolution strategies. There are 129 (68%) examined issues concerning debugging, but existing tools can detect at most 14 issues and fix none. It implies that existing tools provide limited assistance in automation configuration. Our research sheds light on future directions in smart home development.