Context-aware LLM-based AI Agents for Human-centered Energy Management Systems in Smart Buildings

TL;DR

The paper introduces a context-aware, LLM-based ambient AI agent for human-centered BEMS in smart buildings, organized into perception, brain, and action modules. A residential prototype evaluates feasibility and generalizability across four real-world datasets using 120 queries spanning 6 primary and 24 secondary intents, reporting an average latency of about $23$ seconds and an overall accuracy near $0.79$, with stronger performance in device control and energy analysis than in cost management. A formal ANOVA-based generalizability assessment shows consistency across buildings, highlighting the potential of ambient AI agents to ground energy guidance in building-specific data and constraints. The study provides a benchmarking framework and discusses tradeoffs between accuracy and efficiency, underscoring implications for real-world deployment and future work on latency reduction, safety/privacy, and multi-agent architectures.

Abstract

This study presents a conceptual framework and a prototype assessment for Large Language Model (LLM)-based Building Energy Management System (BEMS) AI agents to facilitate context-aware energy management in smart buildings through natural language interaction. The proposed framework comprises three modules: perception (sensing), central control (brain), and action (actuation and user interaction), forming a closed feedback loop that captures, analyzes, and interprets energy data to respond intelligently to user queries and manage connected appliances. By leveraging the autonomous data analytics capabilities of LLMs, the BEMS AI agent seeks to offer context-aware insights into energy consumption, cost prediction, and device scheduling, thereby addressing limitations in existing energy management systems. The prototype's performance was evaluated using 120 user queries across four distinct real-world residential energy datasets and different evaluation metrics, including latency, functionality, capability, accuracy, and cost-effectiveness. The generalizability of the framework was demonstrated using ANOVA tests. The results revealed promising performance, measured by response accuracy in device control (86%), memory-related tasks (97%), scheduling and automation (74%), and energy analysis (77%), while more complex cost estimation tasks highlighted areas for improvement with an accuracy of 49%. This benchmarking study moves toward formalizing the assessment of LLM-based BEMS AI agents and identifying future research directions, emphasizing the trade-off between response accuracy and computational efficiency.

Figures (7)

• Figure 1: The conceptual framework of the LLM-based Human-centered BEMS (Building Energy Management System) AI Agent
• Figure 2: The prototype design and operation flow of the LLM-based Building Energy Management System (BEMS) AI Agent
• Figure 3: Boxplots distribution of the evaluation metrics for different primary categories of the user queries
• Figure 4: Heatmap distribution of the evaluation metrics for different categories of tested user queries
• Figure 5: Examples of workflow of the AI agent responding to user queries