Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An LLM-Based Digital Twin for Optimizing Human-in-the Loop Systems

Hanqing Yang, Marie Siew, Carlee Joe-Wong

TL;DR

This paper presents a case study that employs large language models (LLM) agents to mimic the behaviors and thermal preferences of various population groups in a shopping mall, and shows that LLMs are capable of simulating complex population movements within large open spaces.

Abstract

The increasing prevalence of Cyber-Physical Systems and the Internet of Things (CPS-IoT) applications and Foundation Models are enabling new applications that leverage real-time control of the environment. For example, real-time control of Heating, Ventilation and Air-Conditioning (HVAC) systems can reduce its usage when not needed for the comfort of human occupants, hence reducing energy consumption. Collecting real-time feedback on human preferences in such human-in-the-loop (HITL) systems, however, is difficult in practice. We propose the use of large language models (LLMs) to deal with the challenges of dynamic environments and difficult-to-obtain data in CPS optimization. In this paper, we present a case study that employs LLM agents to mimic the behaviors and thermal preferences of various population groups (e.g. young families, the elderly) in a shopping mall. The aggregated thermal preferences are integrated into an agent-in-the-loop based reinforcement learning algorithm AitL-RL, which employs the LLM as a dynamic simulation of the physical environment to learn how to balance between energy savings and occupant comfort. Our results show that LLMs are capable of simulating complex population movements within large open spaces. Besides, AitL-RL demonstrates superior performance compared to the popular existing policy of set point control, suggesting that adaptive and personalized decision-making is critical for efficient optimization in CPS-IoT applications. Through this case study, we demonstrate the potential of integrating advanced Foundation Models like LLMs into CPS-IoT to enhance system adaptability and efficiency. The project's code can be found on our GitHub repository.

An LLM-Based Digital Twin for Optimizing Human-in-the Loop Systems

TL;DR

This paper presents a case study that employs large language models (LLM) agents to mimic the behaviors and thermal preferences of various population groups in a shopping mall, and shows that LLMs are capable of simulating complex population movements within large open spaces.

Abstract

The increasing prevalence of Cyber-Physical Systems and the Internet of Things (CPS-IoT) applications and Foundation Models are enabling new applications that leverage real-time control of the environment. For example, real-time control of Heating, Ventilation and Air-Conditioning (HVAC) systems can reduce its usage when not needed for the comfort of human occupants, hence reducing energy consumption. Collecting real-time feedback on human preferences in such human-in-the-loop (HITL) systems, however, is difficult in practice. We propose the use of large language models (LLMs) to deal with the challenges of dynamic environments and difficult-to-obtain data in CPS optimization. In this paper, we present a case study that employs LLM agents to mimic the behaviors and thermal preferences of various population groups (e.g. young families, the elderly) in a shopping mall. The aggregated thermal preferences are integrated into an agent-in-the-loop based reinforcement learning algorithm AitL-RL, which employs the LLM as a dynamic simulation of the physical environment to learn how to balance between energy savings and occupant comfort. Our results show that LLMs are capable of simulating complex population movements within large open spaces. Besides, AitL-RL demonstrates superior performance compared to the popular existing policy of set point control, suggesting that adaptive and personalized decision-making is critical for efficient optimization in CPS-IoT applications. Through this case study, we demonstrate the potential of integrating advanced Foundation Models like LLMs into CPS-IoT to enhance system adaptability and efficiency. The project's code can be found on our GitHub repository.
Paper Structure (8 sections, 4 equations, 6 figures, 1 table)

This paper contains 8 sections, 4 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Agent-in-the-Loop Learning
  3. Environment Setup
  4. Reinforcement Learning
  5. Evaluation and Results
  6. Experiment Design
  7. Reinforcement Learning Performance
  8. Discussion and Conclusion

Figures (6)

  • Figure 1: The LLM-based Digital Twin Agent in the Loop Distributed Control (AitL-RL) Pipeline. The LLM-based digital twin simulates population behavior in the mall across the day, with multiple population groups such as "teen shoppers". Based on the simulation, user preferences are aggregated and input into the Agent-in-the-loop RL algorithm for offline training to optimize user comfort and energy savings.
  • Figure 2: The layout of the shopping mall named Happy Mall, whose floor plan was generated by GPT 4, after minor manual adjustments to better replicate a real mall setting.
  • Figure 3: Simulation of movement in Happy Mall over a day. These distributions for the digital twin are generated by GPT 3.5 in each time slot. The population groups exhibit different temporal patterns (results averaged over 10 experiments).
  • Figure 4: Comparison of training convergence using different methods: balanced weights (considering both user comfort and energy cost), energy-focused, and user-comfort-focused. All settings converge, and the balanced weights approach achieves the best overall score, emphasizing the need for equal consideration of multiple aspects.
  • Figure 5: Comparison of policy performance in an online scenario using a balanced weights approach. The AitL-RL policies outperform the set-point policy, and distributed control is more efficient than centralized control.
  • ...and 1 more figures