Enhancing Smart Environments with Context-Aware Chatbots using Large Language Models

TL;DR

The paper presents a context-aware chatbot architecture for smart environments by fusing HAR with UWB-based indoor localization and LLM-powered dialogue. Ambient sensors collect data, HAR interprets activities, and UWB locates users; these contexts feed a Gemini-based chatbot to generate personalized, proactive interactions and control smart-home actuators. A case study in a supervised flat with three frail adults demonstrates feasibility and benefits of real-time, context-driven conversations. The approach emphasizes privacy via edge/fog processing and a modular prompt design that preserves coherent dialogue over time. Findings suggest practical impact for independent living and potential deployment in assisted living facilities.

Abstract

This work presents a novel architecture for context-aware interactions within smart environments, leveraging Large Language Models (LLMs) to enhance user experiences. Our system integrates user location data obtained through UWB tags and sensor-equipped smart homes with real-time human activity recognition (HAR) to provide a comprehensive understanding of user context. This contextual information is then fed to an LLM-powered chatbot, enabling it to generate personalised interactions and recommendations based on the user's current activity and environment. This approach moves beyond traditional static chatbot interactions by dynamically adapting to the user's real-time situation. A case study conducted from a real-world dataset demonstrates the feasibility and effectiveness of our proposed architecture, showcasing its potential to create more intuitive and helpful interactions within smart homes. The results highlight the significant benefits of integrating LLM with real-time activity and location data to deliver personalised and contextually relevant user experiences.

Figures (3)

• Figure 1: Basic components for multi-occupancy activity recognition based on user location and object nearby interaction.
• Figure 2: Basic components for integrating chatbots for Context-Aware User Interaction from HAR using LLMs
• Figure 3: Room distribution in the apartment: Living Room, Office, Bedroom 1, Kitchen, Bathroom, and Bedroom 2.