Physiological and Behavioral Modeling of Stress and Cognitive Load in Web-Based Question Answering

TL;DR

This study addresses the challenge of detecting rapid cognitive load and stress during short web-based question-answer tasks by combining a $2\times2$ within-subjects design with multimodal sensing (mouse dynamics, eye-tracking, EDA, ECG). Its core contributions are: (1) empirical mapping of how objective task demands and subjective experience relate to fine-grained psychophysiological and behavioral signals, (2) demonstration that machine learning models can classify cognitive load and stress from multimodal data with meaningful accuracy, and (3) a tiered, user-centric framework for adaptive survey interfaces that uses signal modalities to determine when and how to intervene. The findings show both convergences and divergences between design intent and lived experience, highlighting the importance of incorporating subjective appraisals into adaptive systems. Technically, response time and tonic EDA emerge as key discriminators, with SHAP analyses illustrating modality-specific contributions and the added value of combining behavioral and physiological data to capture nuanced states in sub-$20$-second windows. Practically, the work lays the groundwork for ethical, transparent, and user-controlled adaptive survey designs that respond to sustained or transient strain without disrupting task flow. Open science practices further bolster reproducibility and impact in real-world web survey contexts.

Abstract

Time pressure and question difficulty can trigger stress and cognitive overload in web-based surveys, compromising data quality and user experience. Most stress detection methods are based on low-resolution self-reports, which are poorly suited for capturing fast, moment-to-moment changes during short online tasks. Addressing this gap, we conducted a 2x2 within-subjects study (N = 29), manipulating question difficulty and time pressure in a web-based multiple-choice task. Participants completed general knowledge and cognitive questions while we collected multimodal data: mouse dynamics, eye tracking, electrocardiogram, and electrodermal activity. Using condition-based and self-reported labels, we used statistical and machine learning models to model stress and question difficulty. Our results show distinct physiological and behavioral patterns within very short timeframes. This work demonstrates the feasibility of rapidly detecting cognitive-affective states in digital environments, paving the way for more adaptive, ethical, and user-aware survey interfaces.

Figures (5)

• Figure 1: Flow diagram illustrating the procedure of trials (questions). (1) fixation cross, (2) Multiple-choice question with timer or no timer, (3) Self-report on question difficulty, subjective stress, and confidence in the provided answer, (4) Suggested 20-second rest period with countdown
• Figure 2: The (a) average accuracy and (b) the average response time across the four experimental conditions with standard errors formed by the interaction of question difficulty and time pressure. Accuracy was higher for easy tasks (0.69 without timer, 0.68 with timer) compared to difficult tasks (0.52 without timer, 0.51 with timer). Response times were consistently longer without time pressure, indicating that both question difficulty and time constraints impacted response behavior.
• Figure 3: SHapley Additive ex-Planations (SHAP) for all features used in the RF classifier, including 4-self-reported-class labels
• Figure 4: Mean and Standard Deviation of the self-reported ratings over manipulated conditions
• Figure 5: Visualization of self-reported stress-difficulty ratings. (a) Simple linear regression shows the overall trend; (b) Piecewise regression with split at difficulty = 4 (Neutral) demonstrates changing slopes across ranges.