Exploration of LLMs, EEG, and behavioral data to measure and support attention and sleep

TL;DR

The study assesses whether large language models can detect attention and sleep states from biobehavioral data and provide adaptive sleep-support content. It combines EEG, actigraphy, and PSQI-derived information across two experiments: state detection (attention, sleep stages, sleep quality) and personalized sleep-improvement feedback (suggestions and guided imagery). Traditional ML generally outperforms LLMs in state detection, though fine-tuned GPT-3.5 shows best LLM performance, while GPT-4 vision faces limitations; sleep quality estimation via PSQI text achieves strong accuracy with GPT-4. Additionally, LLMs can generate CBT-I-aligned, personalized content, but results hinge on prompt design and safety considerations. The work highlights both the promise and current constraints of integrating LLMs with multimodal biosignals for adaptive health interventions, underscoring the need for larger diverse datasets and robust multimodal fusion approaches.

Abstract

We explore the application of large language models (LLMs), pre-trained models with massive textual data for detecting and improving these altered states. We investigate the use of LLMs to estimate attention states, sleep stages, and sleep quality and generate sleep improvement suggestions and adaptive guided imagery scripts based on electroencephalogram (EEG) and physical activity data (e.g. waveforms, power spectrogram images, numerical features). Our results show that LLMs can estimate sleep quality based on human textual behavioral features and provide personalized sleep improvement suggestions and guided imagery scripts; however detecting attention, sleep stages, and sleep quality based on EEG and activity data requires further training data and domain-specific knowledge.

Figures (2)

• Figure 1: Image Data Fed into LLMs
• Figure 2: Sleep stage detection models: confusion matrix