Sensing What Surveys Miss: Understanding and Personalizing Proactive LLM Support by User Modeling

TL;DR

The paper tackles cognitive overload and poor help-seeking in sequential knowledge tasks by introducing a proactive, personalized LLM assistant that uses multimodal sensing (EDA and mouse dynamics) to predict moment-to-moment support needs. It develops a dual-modality classifier with per-user threshold adaptation, triggering LLM-based clarifications only when cognitive load is detected as high, and compares aligned, misaligned, and random timing in a within-subject study (N=32). Results show aligned adaptive timing improves accuracy from about 41% to 62%, reduces false negatives, and enhances perceived efficiency, dependability, and benevolence, indicating stronger user trust. The work demonstrates that precise, person-specific timing of assistance can preserve data quality and user experience, offering practical implications for survey design, education, and healthcare forms where sequential cognitive demand accumulates.

Abstract

Difficulty spillover and suboptimal help-seeking challenge the sequential, knowledge-intensive nature of digital tasks. In online surveys, tough questions can drain mental energy and hurt performance on later questions, while users often fail to recognize when they need assistance or may satisfy, lacking motivation to seek help. We developed a proactive, adaptive system using electrodermal activity and mouse movement to predict when respondents need support. Personalized classifiers with a rule-based threshold adaptation trigger timely LLM-based clarifications and explanations. In a within-subjects study (N=32), aligned-adaptive timing was compared to misaligned-adaptive and random-adaptive controls. Aligned-adaptive assistance improved response accuracy by 21%, reduced false negative rates from 50.9% to 22.9%, and improved perceived efficiency, dependability, and benevolence. Properly timed interventions prevent cascades of degraded responses, showing that aligning support with cognitive states improves both the outcomes and the user experience. This enables more effective, personalized LLM-assisted support in survey-based research.

Figures (9)

• Figure 1: Screenshots of the experimental interface: (a) calibration phase, (b) rest break, (c) condition task question, (d) adaptive helper triggered, (e) waiting for user selecting text, and (f) assistance provided by the system.
• Figure 2: Rule-based threshold adaptation logic per trial: positive changes increase the threshold (fewer future interventions), negative changes decrease it (more future interventions).
• Figure 3: The task was structured in blocks. First, participants completed a personalization block to calibrate physiological and behavioral signals. This was followed by three condition blocks, Aligned-Adaptive, Misaligned-Adaptive, and Random, presented in randomized order (×3). After each condition block, participants provided evaluation ratings, and the session concluded with a post-study interview.
• Figure 4: Confusion matrices showing model performance in knowing whether users need assistance across the three experimental conditions (Aligned-Adaptive, Misaligned-Adaptive, and Random-Adaptive). Each matrix compares the system's predicted cognitive state with the participants' self-reported need, illustrating accuracy, false positives, and false negatives per condition.
• Figure 5: Box plots showing rates of helpers (1) shown, (2) accepted, and (3) acceptance rate distributions across the three experimental conditions (Aligned-Adaptive, Misaligned-Adaptive, and Random-Adaptive).