Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Game of LLMs: Discovering Structural Constructs in Activities using Large Language Models

Shruthi K. Hiremath, Thomas Ploetz

TL;DR

This work focuses on identifying these underlying building blocks--structural constructs, with the use of large language models, that can be beneficial especially in recognizing short-duration and infrequent activities, which current systems cannot recognize.

Abstract

Human Activity Recognition is a time-series analysis problem. A popular analysis procedure used by the community assumes an optimal window length to design recognition pipelines. However, in the scenario of smart homes, where activities are of varying duration and frequency, the assumption of a constant sized window does not hold. Additionally, previous works have shown these activities to be made up of building blocks. We focus on identifying these underlying building blocks--structural constructs, with the use of large language models. Identifying these constructs can be beneficial especially in recognizing short-duration and infrequent activities. We also propose the development of an activity recognition procedure that uses these building blocks to model activities, thus helping the downstream task of activity monitoring in smart homes.

Game of LLMs: Discovering Structural Constructs in Activities using Large Language Models

TL;DR

This work focuses on identifying these underlying building blocks--structural constructs, with the use of large language models, that can be beneficial especially in recognizing short-duration and infrequent activities, which current systems cannot recognize.

Abstract

Human Activity Recognition is a time-series analysis problem. A popular analysis procedure used by the community assumes an optimal window length to design recognition pipelines. However, in the scenario of smart homes, where activities are of varying duration and frequency, the assumption of a constant sized window does not hold. Additionally, previous works have shown these activities to be made up of building blocks. We focus on identifying these underlying building blocks--structural constructs, with the use of large language models. Identifying these constructs can be beneficial especially in recognizing short-duration and infrequent activities. We also propose the development of an activity recognition procedure that uses these building blocks to model activities, thus helping the downstream task of activity monitoring in smart homes.
Paper Structure (13 sections, 2 figures, 2 tables)

This paper contains 13 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Temporal Constructs in Sequential Data
  4. Discovering Structural Concepts in Activities using LLMs
  5. Large Language Models
  6. GPT-4: Activity summarization
  7. Gemini: Identifying activity specific structural constructs
  8. Experimental Evaluation
  9. Datasets and Data Pre-Processing
  10. Determining Structural Concepts
  11. Utility of Identified Constructs: Activity Recognition
  12. Discussion and Future Work
  13. Conclusion

Figures (2)

  • Figure 1: Overview of the proposed system. The proposed approach identifies the underlying structural concepts of activities observed in the smart home. First sentences detailing sensor event triggers are generated using information such as location and time of occurrence of activity thukral2024layout. Next a family of LLMs (GPT-4) is used to obtain a summarized version of varied instances of these activities. Subsequently, another family of LLMs (Gemini) is queried to identify the structural constructs.
  • Figure 2: Floor plans for Smart Homes used for our experimental evaluation: (a) CASAS-Aruba and (b) CASAS-Milan (taken with permission from cook2012casas). Annotations for locations are used with permission from hiremath2022bootstrapping.