Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Extending Stress Detection Reproducibility to Consumer Wearable Sensors

Ohida Binte Amin, Varun Mishra, Tinashe M. Tapera, Robert Volpe, Aarti Sathyanarayana

TL;DR

This study tackles the reproducibility gap in stress-detection models by extending cross-device evaluation to consumer wearables. It introduces a cross-device stress-prediction tool tested on four devices—Biopac MP160, Polar H10, Empatica E4, and Garmin Forerunner 55s—using HRV and EDA with LOSO validation and cross-study comparison, including a 35-participant in-lab stress protocol. Results show that combining HRV and EDA yields the best performance overall, with Biopac MP160 achieving high AUROC (up to $0.984$) and Garmin Forerunner 55s achieving strong performance (up to $0.961$), while Empatica E4 exhibits variability and limited generalization across deployments. The findings support the feasibility of consumer wearables for real-world stress monitoring while highlighting device-specific data-quality issues and generalization challenges that shape practical digital phenotyping efforts; future work should expand device sets, populations, and in-the-wild validation to improve generalizability.

Abstract

Wearable sensors are widely used to collect physiological data and develop stress detection models. However, most studies focus on a single dataset, rarely evaluating model reproducibility across devices, populations, or study conditions. We previously assessed the reproducibility of stress detection models across multiple studies, testing models trained on one dataset against others using heart rate (with R-R interval) and electrodermal activity (EDA). In this study, we extended our stress detection reproducibility to consumer wearable sensors. We compared validated research-grade devices, to consumer wearables - Biopac MP160, Polar H10, Empatica E4, to the Garmin Forerunner 55s, assessing device-specific stress detection performance by conducting a new stress study on undergraduate students. Thirty-five students completed three standardized stress-induction tasks in a lab setting. Biopac MP160 performed the best, being consistent with our expectations of it as the gold standard, though performance varied across devices and models. Combining heart rate variability (HRV) and EDA enhanced stress prediction across most scenarios. However, Empatica E4 showed variability; while HRV and EDA improved stress detection in leave-one-subject-out (LOSO) evaluations (AUROC up to 0.953), device-specific limitations led to underperformance when tested with our pre-trained stress detection tool (AUROC 0.723), highlighting generalizability challenges related to hardware-model compatibility. Garmin Forerunner 55s demonstrated strong potential for real-world stress monitoring, achieving the best mental arithmetic stress detection performance in LOSO (AUROC up to 0.961) comparable to research-grade devices like Polar H10 (AUROC 0.954), and Empatica E4 (AUROC 0.905 with HRV-only model and AUROC 0.953 with HRV+EDA model), with the added advantage of consumer-friendly wearability for free-living contexts.

Extending Stress Detection Reproducibility to Consumer Wearable Sensors

TL;DR

This study tackles the reproducibility gap in stress-detection models by extending cross-device evaluation to consumer wearables. It introduces a cross-device stress-prediction tool tested on four devices—Biopac MP160, Polar H10, Empatica E4, and Garmin Forerunner 55s—using HRV and EDA with LOSO validation and cross-study comparison, including a 35-participant in-lab stress protocol. Results show that combining HRV and EDA yields the best performance overall, with Biopac MP160 achieving high AUROC (up to ) and Garmin Forerunner 55s achieving strong performance (up to ), while Empatica E4 exhibits variability and limited generalization across deployments. The findings support the feasibility of consumer wearables for real-world stress monitoring while highlighting device-specific data-quality issues and generalization challenges that shape practical digital phenotyping efforts; future work should expand device sets, populations, and in-the-wild validation to improve generalizability.

Abstract

Wearable sensors are widely used to collect physiological data and develop stress detection models. However, most studies focus on a single dataset, rarely evaluating model reproducibility across devices, populations, or study conditions. We previously assessed the reproducibility of stress detection models across multiple studies, testing models trained on one dataset against others using heart rate (with R-R interval) and electrodermal activity (EDA). In this study, we extended our stress detection reproducibility to consumer wearable sensors. We compared validated research-grade devices, to consumer wearables - Biopac MP160, Polar H10, Empatica E4, to the Garmin Forerunner 55s, assessing device-specific stress detection performance by conducting a new stress study on undergraduate students. Thirty-five students completed three standardized stress-induction tasks in a lab setting. Biopac MP160 performed the best, being consistent with our expectations of it as the gold standard, though performance varied across devices and models. Combining heart rate variability (HRV) and EDA enhanced stress prediction across most scenarios. However, Empatica E4 showed variability; while HRV and EDA improved stress detection in leave-one-subject-out (LOSO) evaluations (AUROC up to 0.953), device-specific limitations led to underperformance when tested with our pre-trained stress detection tool (AUROC 0.723), highlighting generalizability challenges related to hardware-model compatibility. Garmin Forerunner 55s demonstrated strong potential for real-world stress monitoring, achieving the best mental arithmetic stress detection performance in LOSO (AUROC up to 0.961) comparable to research-grade devices like Polar H10 (AUROC 0.954), and Empatica E4 (AUROC 0.905 with HRV-only model and AUROC 0.953 with HRV+EDA model), with the added advantage of consumer-friendly wearability for free-living contexts.
Paper Structure (14 sections, 2 figures, 2 tables)

This paper contains 14 sections, 2 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. Methodology
  3. Approach & Development of Stress Prediction Tool for Evaluating Reproducibility
  4. Physiological Signal Selection & Pre-processing
  5. Feature Extraction
  6. Machine Learning Models and Evaluation
  7. Extending Stress Detection Reproducibility with Multi-Device Testing
  8. In-Lab Calibration and Stress Induction Protocol
  9. Comparative Analysis of Stress Detection Performance Across Devices
  10. Results
  11. Socio-demographics of Participants in Digital Phenotyping Stress Study
  12. Comparative Stress Prediction Reproducibility Performance Across Devices
  13. DISCUSSIONS
  14. CONCLUSIONS

Figures (2)

  • Figure 1: Lab protocol timeline for three stressor conditions in our digital phenotyping stress study, where the first two stress tasks were randomized. The cold pressor was conducted at the end due to logistical constraints.
  • Figure 2: Stress detection performance comparison of consumer wearable, Garmin Forerunner 55s and the research-grade Polar H10 in a lab setting with minimal differences in AUROC scores. Garmin Forerunner 55s offers greater flexibility for real-world use due to its wrist-worn design, whereas Polar H10's chest strap positioning may limit wearability and convenience for long-term stress monitoring.