Understanding State Social Anxiety in Virtual Social Interactions using Multimodal Wearable Sensing Indicators

TL;DR

This study tackles minute-scale detection of state social anxiety during virtual interactions by combining wearable biosignals (PPG, EDA, accelerometer, skin temperature), contextual social features (group size, anticipatory/concurrent/post phases, evaluative threat), and stable trait measures. Using nested cross-validation and LOOCV, biobehavioral data alone achieve modest detection of anxiety states, but integration with social-context and trait information substantially boosts accuracy, particularly for extreme anxiety elevations. Key findings show PPG-derived HRV features are strong predictors, contextual factors (e.g., number of partners, phase timing) substantially improve models, and trait measures contribute additional gains, with best-performing models reaching around $69 o84egin{cases} ext{balanced accuracy} \\ ext{extreme anxiety detection} \\ ext{(overall)} \\ \\end{cases}$ depending on the outcome. The results underscore the importance of multimodal data and context-aware modeling for real-time mental-health monitoring and lay groundwork for future in-context interventions and JITAI systems in both virtual and potentially in-person settings.

Abstract

Mobile sensing is ubiquitous and offers opportunities to gain insight into state mental health functioning. Detecting state elevations in social anxiety would be especially useful given this phenomenon is highly prevalent and impairing, but often not disclosed. Although anxiety is highly dynamic, fluctuating rapidly over the course of minutes, most work to date has examined anxiety at a scale of hours, days, or longer. In the present work, we explore the feasibility of detecting fluctuations in state social anxiety among N = 46 undergraduate students with elevated symptoms of trait social anxiety. Participants engaged in two dyadic and two group social interactions via Zoom. We evaluated participants' state anxiety levels as they anticipated, immediately after experiencing, and upon reflecting on each social interaction, spanning a time frame of 2-6 minutes. We collected biobehavioral features (i.e., PPG, EDA, skin temperature, and accelerometer) via Empatica E4 devices as they participated in the varied social contexts (e.g., dyadic vs. group; anticipating vs. experiencing the interaction; experiencing varying levels of social evaluation). We additionally measured their trait mental health functioning. Mixed-effect logistic regression and leave-one-subject-out machine learning modeling indicated biobehavioral features significantly predict state fluctuations in anxiety, though balanced accuracy tended to be modest (59%). However, our capacity to identify instances of heightened versus low state anxiety significantly increased (with balanced accuracy ranging from 69% to 84% across different operationalizations of state anxiety) when we integrated contextual data alongside trait mental health functioning into our predictive models.. We discuss these and other findings in the context of the broader anxiety detection literature.

Figures (7)

• Figure 1: Composite View of State Anxiety Distributions: (a) Sample-wide distribution of raw scores, (b) adjusted within-person score variability, and (c) scores across individual participants.
• Figure 2: Comparative Analysis of Anxiety Levels Across Different Social Contexts. This figure presents comparisons of anxiety levels using the paired samples Wilcoxon test across various social experiences (a), phases of interaction (b), group contexts (c), and degrees of evaluative threat (d). The dashed lines within each violent plot delineate the four quartiles. The Benjamini-Hochberg correction was applied to address multiple comparisons within each group of contextual variables in each subfigure, with significance brackets and asterisks marking pairs that demonstrated statistically significant differences at $p<0.05$.
• Figure 3: Results of ablation experiments examining the impact of contextual and trait measures on model performance alongside biobehavioral features. The error bars quantify the standard deviation of the model performance across 10 training and evaluation experiments.
• Figure 4: Comparative analysis of model performance: sensor-level contributions (left) and feature-level significance within sensors (right) in ablation studies. The error bars in both figures represent the variability in model performance across different model iterations for each participant individually.
• Figure 5: Scatter plots with Pearson-correlation lines illustrating the relationship between sample-wide individual-level model accuracies, trait anxiety SIAS scores, and means and standard deviations of state anxiety.