Leveraging Large Language Models for Generating Mobile Sensing Strategies in Human Behavior Modeling
Nan Gao, Zhuolei Yu, Yue Xu, Chun Yu, Yuntao Wang, Flora D. Salim, Yuanchun Shi
TL;DR
Mobile sensing offers rich, passively collected data for understanding daily human behavior, but designing sensing strategies—sensor choice, data collection, and feature construction—remains labor-intensive and poorly generalizable. The authors introduce an automated mobile sensing strategy generation framework that builds a knowledge base from 55 prior studies, employs a multi-granular representation of behaviour, and uses Large Language Models to generate tailored sensing strategies through a structured five-step workflow guided by a prompt design. Key contributions include the knowledge base of feature-construction and sensor-selection rules, the four-level behaviour representation, and an automated strategy generation system with data-source recommendations, feature construction, model guidance, and performance estimates, validated by expert-driven comparative and usability evaluations. This approach promises to reduce research burden, enhance cross-scenario applicability, and support privacy-aware, on-device automation to advance health and well-being applications through mobile sensing.
Abstract
Mobile sensing plays a crucial role in generating digital traces to understand human daily lives. However, studying behaviours like mood or sleep quality in smartphone users requires carefully designed mobile sensing strategies such as sensor selection and feature construction. This process is time-consuming, burdensome, and requires expertise in multiple domains. Furthermore, the resulting sensing framework lacks generalizability, making it difficult to apply to different scenarios. In the research, we propose an automated mobile sensing strategy for human behaviour understanding. First, we establish a knowledge base and consolidate rules for data collection and effective feature construction. Then, we introduce the multi-granular human behaviour representation and design procedures for leveraging large language models to generate strategies. Our approach is validated through blind comparative studies and usability evaluation. Ultimately, our approach holds the potential to revolutionise the field of mobile sensing and its applications.