Leveraging Large Language Models for enhanced personalised user experience in Smart Homes

TL;DR

This work tackles the challenge of achieving highly personalized smart-home automation without extensive hand-crafted routines. It proposes a central decision loop powered by large language models, enhanced with retrieval-augmented generation and explicit user preferences, plus a natural-language representation of the home state. Key findings show that incorporating preferences can yield substantial gains in alignment with user desires (up to about 52% in average grade), and that smaller open-source models can perform competitively when paired with natural-language representations, often with faster inference than larger models. The approach demonstrates the practicality of proactive, personalized smart homes, while acknowledging non-determinism and latency as key challenges for real-world deployment.

Abstract

Smart home automation systems aim to improve the comfort and convenience of users in their living environment. However, adapting automation to user needs remains a challenge. Indeed, many systems still rely on hand-crafted routines for each smart object.This paper presents an original smart home architecture leveraging Large Language Models (LLMs) and user preferences to push the boundaries of personalisation and intuitiveness in the home environment.This article explores a human-centred approach that uses the general knowledge provided by LLMs to learn and facilitate interactions with the environment.The advantages of the proposed model are demonstrated on a set of scenarios, as well as a comparative analysis with various LLM implementations. Some metrics are assessed to determine the system's ability to maintain comfort, safety, and user preferences. The paper details the approach to real-world implementation and evaluation.The proposed approach of using preferences shows up to 52.3% increase in average grade, and with an average processing time reduced by 35.6% on Starling 7B Alpha LLM. In addition, performance is 26.4% better than the results of the larger models without preferences, with processing time almost 20 times faster.

Figures (3)

• Figure 1: The implemented architecture
• Figure 2: Extract of natural data representation on the scenario "Out of bed at night"
• Figure 3: Average grades by model, data representation and prompting styles