Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating a VR System for Collecting Safety-Critical Vehicle-Pedestrian Interactions

Erica Weng, Kenta Mukoya, Deva Ramanan, Kris Kitani

TL;DR

The paper addresses the scarcity of safety-critical pedestrian trajectory data for autonomous vehicles due to real-world ethical and logistical constraints. It proposes a VR-based data-collection system integrated with CARLA to capture realistic pedestrian trajectories and full-body pose in controlled, low-risk environments, validated through semi-structured interviews and three empirical studies (a first-person study with 62–64 participants and a third-person realism evaluation with 302 respondents). Results show VR data elicits realistic pedestrian responses; third-party evaluators rate VR trajectories as closer to real-life data than fully synthetic trajectories, supporting VR as a viable supplement to real-world datasets. The work demonstrates VR can mitigate sim-to-real gaps, enable collection of uncommon and vulnerable-road-user scenarios, and provide rich pose information, offering a scalable approach for improving AV safety models while addressing ethical concerns.

Abstract

Autonomous vehicles (AVs) require comprehensive and reliable pedestrian trajectory data to ensure safe operation. However, obtaining data of safety-critical scenarios such as jaywalking and near-collisions, or uncommon agents such as children, disabled pedestrians, and vulnerable road users poses logistical and ethical challenges. This paper evaluates a Virtual Reality (VR) system designed to collect pedestrian trajectory and body pose data in a controlled, low-risk environment. We substantiate the usefulness of such a system through semi-structured interviews with professionals in the AV field, and validate the effectiveness of the system through two empirical studies: a first-person user evaluation involving 62 participants, and a third-person evaluative survey involving 290 respondents. Our findings demonstrate that the VR-based data collection system elicits realistic responses for capturing pedestrian data in safety-critical or uncommon vehicle-pedestrian interaction scenarios.

Evaluating a VR System for Collecting Safety-Critical Vehicle-Pedestrian Interactions

TL;DR

The paper addresses the scarcity of safety-critical pedestrian trajectory data for autonomous vehicles due to real-world ethical and logistical constraints. It proposes a VR-based data-collection system integrated with CARLA to capture realistic pedestrian trajectories and full-body pose in controlled, low-risk environments, validated through semi-structured interviews and three empirical studies (a first-person study with 62–64 participants and a third-person realism evaluation with 302 respondents). Results show VR data elicits realistic pedestrian responses; third-party evaluators rate VR trajectories as closer to real-life data than fully synthetic trajectories, supporting VR as a viable supplement to real-world datasets. The work demonstrates VR can mitigate sim-to-real gaps, enable collection of uncommon and vulnerable-road-user scenarios, and provide rich pose information, offering a scalable approach for improving AV safety models while addressing ethical concerns.

Abstract

Autonomous vehicles (AVs) require comprehensive and reliable pedestrian trajectory data to ensure safe operation. However, obtaining data of safety-critical scenarios such as jaywalking and near-collisions, or uncommon agents such as children, disabled pedestrians, and vulnerable road users poses logistical and ethical challenges. This paper evaluates a Virtual Reality (VR) system designed to collect pedestrian trajectory and body pose data in a controlled, low-risk environment. We substantiate the usefulness of such a system through semi-structured interviews with professionals in the AV field, and validate the effectiveness of the system through two empirical studies: a first-person user evaluation involving 62 participants, and a third-person evaluative survey involving 290 respondents. Our findings demonstrate that the VR-based data collection system elicits realistic responses for capturing pedestrian data in safety-critical or uncommon vehicle-pedestrian interaction scenarios.
Paper Structure (27 sections, 6 figures, 5 tables)

This paper contains 27 sections, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Human Behavior in Virtual Environments
  4. Pedestrian Behavior Studies
  5. 2D Pedestrian Trajectory Data Capture: Real-Life, Human-Informed, and Fully-Synthetic
  6. Part 1: Understanding Limitations of Existing Vehicle-Pedestrian Datasets
  7. Semi-Structured Interview Recruitment and Design
  8. Interview Responses
  9. Lack of Uncommon but Important Scenarios
  10. Lack of Interesting Vehicle-Pedestrian Interactions
  11. Lack of Fine-grained Trajectory Features
  12. Summary
  13. VR Pedestrian Trajectory and Pose Collection System Overview
  14. Part 2: VR Sense of Presence User Study
  15. Study Design
  16. ...and 12 more sections

Figures (6)

  • Figure 1: Left: Participant's avatar within the virtual environment from a third-person perspective. Right: participant wearing the VR headset in the data collection environment.
  • Figure 2: Schematic of the VR data collection system.
  • Figure 3: Left: The VR system set up in the classroom data collection space. Right: A pedestrian subject wearing the VR headset.
  • Figure 4: The three tasks that we ask our study participants to complete. Top: snapshot of each task as seen in the CARLA virtual environment. Bottom: schematic of each task; "S" denotes participant start location and "G" denotes goal. Schematic images also used in anon2023.
  • Figure 5: Various study participants wearing the VR headset and walking around in two different classroom-setting data collection environments.
  • ...and 1 more figures