Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Augmented Reality Warnings in Roadway Work Zones: Evaluating the Effect of Modality on Worker Reaction Times

Sepehr Sabeti, Fatemeh Banani Ardecani, Omidreza Shoghli

TL;DR

This study examines how multimodal augmented reality warnings influence roadway worker reaction times by integrating an AR prototype, immersive VR simulations, and Wizard of Oz synchronization across five experiments. It compares four warning modalities (Visual, Audio-Visual, Haptic-Visual, and Haptic-Audio-Visual) across real-world field tests, indoor desktops, and VR environments, using simple RT and a vision-based pose-tracking method. Key findings show that HV warnings typically produce the fastest reactions, HAV does not consistently outperform HV, and real-world AR RTs are longer and more variable than VR or desktop baselines. The results support the practical potential of personalized AR warnings to achieve RTs under 0.6 seconds and demonstrate the promise of vision-based RT measurement for non-intrusive worker safety monitoring, while also highlighting VR’s partial fidelity to real-world dynamics and the need for future audio-design exploration.

Abstract

Given the aging highway infrastructure requiring extensive rebuilding and enhancements, and the consequent rise in the number of work zones, there is an urgent need to develop advanced safety systems to protect workers. While Augmented Reality (AR) holds significant potential for delivering warnings to workers, its integration into roadway work zones remains relatively unexplored. The primary objective of this study is to improve safety measures within roadway work zones by conducting an extensive analysis of how different combinations of multimodal AR warnings influence the reaction times of workers. This paper addresses this gap through a series of experiments that aim to replicate the distinctive conditions of roadway work zones, both in real-world and virtual reality environments. Our approach comprises three key components: an advanced AR system prototype, a VR simulation of AR functionality within the work zone environment, and the Wizard of Oz technique to synchronize user experiences across experiments. To assess reaction times, we leverage both the simple reaction time (SRT) technique and an innovative vision-based metric that utilizes real-time pose estimation. By conducting five experiments in controlled outdoor work zones and indoor VR settings, our study provides valuable information on how various multimodal AR warnings impact workers reaction times. Furthermore, our findings reveal the disparities in reaction times between VR simulations and real-world scenarios, thereby gauging VR's capability to mirror the dynamics of roadway work zones. Furthermore, our results substantiate the potential and reliability of vision-based reaction time measurements. These insights resonate well with those derived using the SRT technique, underscoring the viability of this approach for tangible real-world uses.

Augmented Reality Warnings in Roadway Work Zones: Evaluating the Effect of Modality on Worker Reaction Times

TL;DR

This study examines how multimodal augmented reality warnings influence roadway worker reaction times by integrating an AR prototype, immersive VR simulations, and Wizard of Oz synchronization across five experiments. It compares four warning modalities (Visual, Audio-Visual, Haptic-Visual, and Haptic-Audio-Visual) across real-world field tests, indoor desktops, and VR environments, using simple RT and a vision-based pose-tracking method. Key findings show that HV warnings typically produce the fastest reactions, HAV does not consistently outperform HV, and real-world AR RTs are longer and more variable than VR or desktop baselines. The results support the practical potential of personalized AR warnings to achieve RTs under 0.6 seconds and demonstrate the promise of vision-based RT measurement for non-intrusive worker safety monitoring, while also highlighting VR’s partial fidelity to real-world dynamics and the need for future audio-design exploration.

Abstract

Given the aging highway infrastructure requiring extensive rebuilding and enhancements, and the consequent rise in the number of work zones, there is an urgent need to develop advanced safety systems to protect workers. While Augmented Reality (AR) holds significant potential for delivering warnings to workers, its integration into roadway work zones remains relatively unexplored. The primary objective of this study is to improve safety measures within roadway work zones by conducting an extensive analysis of how different combinations of multimodal AR warnings influence the reaction times of workers. This paper addresses this gap through a series of experiments that aim to replicate the distinctive conditions of roadway work zones, both in real-world and virtual reality environments. Our approach comprises three key components: an advanced AR system prototype, a VR simulation of AR functionality within the work zone environment, and the Wizard of Oz technique to synchronize user experiences across experiments. To assess reaction times, we leverage both the simple reaction time (SRT) technique and an innovative vision-based metric that utilizes real-time pose estimation. By conducting five experiments in controlled outdoor work zones and indoor VR settings, our study provides valuable information on how various multimodal AR warnings impact workers reaction times. Furthermore, our findings reveal the disparities in reaction times between VR simulations and real-world scenarios, thereby gauging VR's capability to mirror the dynamics of roadway work zones. Furthermore, our results substantiate the potential and reliability of vision-based reaction time measurements. These insights resonate well with those derived using the SRT technique, underscoring the viability of this approach for tangible real-world uses.
Paper Structure (32 sections, 5 equations, 10 figures, 4 tables)

This paper contains 32 sections, 5 equations, 10 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Safety Measures in Roadway Work Zones
  4. Warnings Modality and Design
  5. Reaction Time: Measures, Influences, and Applications to Roadway Worker Safety
  6. Virtual Reality Simulations for Evaluation of AR Warnings
  7. Methodology
  8. Study Overview
  9. Experimental Apparatus and Setup
  10. Augmented Reality Technology
  11. Virtual Reality Simulation
  12. Desktop-based Setup
  13. Study Design
  14. Warnings
  15. Experiment Procedure and Specifications
  16. ...and 17 more sections

Figures (10)

  • Figure 1: Overview of the Augmented Reality-Based Safety Technology and Its Warning Interface Features
  • Figure 2: Hardware, Software, and Techniques Utilized in Different Approaches for Reaction Time Measurement. (a) A participant interacts with a keyboard, responding to a variety of stimuli (b) participant employing a handheld device to react to stimuli presented via augmented reality glasses. (c) Immersed in a virtual reality environment, a participant uses a controller to respond to delivered stimuli (d) In a virtual work zone environment, a participant operates a leaf blower while cameras capture the body movements.
  • Figure 3: Overview of Designed Multimodal Combination of Warnings and Delivery of Visual Warning
  • Figure 4: Virtual Reality Environment: (a) Lab setup showcasing the participant equipped with a VR headset, physically holding a real-life leaf blower, and wearing a smartwatch. (b) First-person VR perspective depicting the participant approaching the virtual leaf blower, preparing for activation. (c) Immersive and interactive VR environment captured from the participant's viewpoint, demonstrating the successful clearance of the drop inlet (d) Top-down view of the work zone layout, illustrating the operational area and placement of the virtual elements. (e) Near-miss scenario observed from the work zone perspective, showcasing dynamic traffic flow within the simulation
  • Figure 5: Proposed Vision-based Pose Estimation Methodology for Reaction Time Measurement
  • ...and 5 more figures