Metaverse for Safer Roadways: An Immersive Digital Twin Framework for Exploring Human-Autonomy Coexistence in Urban Transportation Systems
Tanmay Vilas Samak, Chinmay Vilas Samak, Venkat Narayan Krovi
TL;DR
The paper tackles the challenge of safely studying human-autonomy coexistence in urban traffic by introducing AutoDRIVE Ecosystem, an immersive digital twin framework with mixed-reality, human-in-the-loop capabilities. It details a modular, open-source platform that connects real and virtual assets in real time, supports V2X communications, and provides interfaces for observation and interaction to study edge-case driving scenarios. Through a mixed-factorial user study and an urban jump-scare case study, the work demonstrates the framework’s ability to assess interface effectiveness, human factors, and autonomous-vehicle behavior in integrated environments. The findings support the framework’s utility for validation and development of socially aware autonomous driving, with practical implications for safety, liability, and human familiarity with autonomous systems.
Abstract
Societal-scale deployment of autonomous vehicles requires them to coexist with human drivers, necessitating mutual understanding and coordination among these entities. However, purely real-world or simulation-based experiments cannot be employed to explore such complex interactions due to safety and reliability concerns, respectively. Consequently, this work presents an immersive digital twin framework to explore and experiment with the interaction dynamics between autonomous and non-autonomous traffic participants. Particularly, we employ a mixed-reality human-machine interface to allow human drivers and autonomous agents to observe and interact with each other for testing edge-case scenarios while ensuring safety at all times. To validate the versatility of the proposed framework's modular architecture, we first present a discussion on a set of user experience experiments encompassing 4 different levels of immersion with 4 distinct user interfaces. We then present a case study of uncontrolled intersection traversal to demonstrate the efficacy of the proposed framework in validating the interactions of a primary human-driven, autonomous, and connected autonomous vehicle with a secondary semi-autonomous vehicle. The proposed framework has been openly released to guide the future of autonomy-oriented digital twins and research on human-autonomy coexistence.