ARCAS: An Augmented Reality Collision Avoidance System with SLAM-Based Tracking for Enhancing VRU Safety

TL;DR

ARCAS tackles VRU safety in mixed traffic by combining a roadside 360° LiDAR with SLAM-based AR headset tracking to deliver world-anchored collision warnings. The system calibrates LiDAR-to-headset geometry automatically and supports multi-headset coordination, rendering 3D bounding boxes and directional arrows in passthrough view. Real-world tests in pedestrian–e-scooter and pedestrian–vehicle scenarios show substantial TTC improvements under AR guidance, either with LiDAR Detection or AR-to-AR sharing. These results validate LiDAR-driven AR guidance as a practical, wearable safety tool and establish a foundation for cooperative AR-based VRU alerts in urban mobility.

Abstract

Vulnerable road users (VRUs) face high collision risks in mixed traffic, yet most existing safety systems prioritize driver or vehicle assistance over direct VRU support. This paper presents ARCAS, a real-time augmented reality collision avoidance system that provides personalized spatial alerts to VRUs via wearable AR headsets. By fusing roadside 360-degree 3D LiDAR with SLAM-based headset tracking and an automatic 3D calibration procedure, ARCAS accurately overlays world-locked 3D bounding boxes and directional arrows onto approaching hazards in the user's passthrough view. The system also enables multi-headset coordination through shared world anchoring. Evaluated in real-world pedestrian interactions with e-scooters and vehicles (180 trials), ARCAS nearly doubled pedestrians' time-to-collision and increased counterparts' reaction margins by up to 4x compared to unaided-eye conditions. Results validate the feasibility and effectiveness of LiDAR-driven AR guidance and highlight the potential of wearable AR as a promising next-generation safety tool for urban mobility.

Figures (3)

• Figure 1: System overview of the proposed ARCAS. The left portion of the figure illustrates the AR headset (a Meta Quest Pro operating in passthrough mode), while the right portion shows the positioning module, consisting of a 3D 360-degree LiDAR sensor and the associated TCP server.
• Figure 2: Overview of the pedestrian–e-scooter collision scenario. (a) The top panel presents a side-view illustration alongside the corresponding real-world scene, in which an AR-equipped pedestrian approaches an intersection while an e-scooter rider—without AR and detected solely by 3D LiDAR—emerges from behind an occluding wall. (b) The bottom panel shows the reconstructed trajectories of the pedestrian and the e-scooter rider, including the points at which each reaches its peak velocity.
• Figure 3: Overview of the pedestrian–vehicle collision scenario. (a) The top panel shows a bird’s-eye illustration alongside the corresponding real-world scene, in which an AR-equipped pedestrian approaches an intersection while a vehicle emerges from an approximately 80$^\circ$ curved and visually occluded side road. (b) The bottom panel illustrates the reconstructed trajectories of both the pedestrian and the vehicle obtained from AR headsets, including the positions at which each reaches its peak velocity.