Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Valeo Near-Field: a novel dataset for pedestrian intent detection

Antonyo Musabini, Rachid Benmokhtar, Jagdish Bhanushali, Victor Galizzi, Bertrand Luvison, Xavier Perrotton

TL;DR

The paper introduces the Valeo Near-Field dataset, a multimodal resource tailored for pedestrian intent detection around a stationary ego-vehicle. It combines synchronized fisheye camera streams, LiDAR, ultrasonic sensors, and motion-capture-based 3D poses to enable robust benchmarking of detection, 3D pose estimation, trajectory, and intent prediction in near-field scenarios. A 300-sequence corpus with a public-test subset and a dedicated benchmark suite provides end-to-end evaluation, including zone-based metrics and occlusion handling. This dataset addresses gaps in near-field perception under occlusion and multi-view conditions, with practical implications for enhancing safety in autonomous driving around pedestrians.

Abstract

This paper presents a novel dataset aimed at detecting pedestrians' intentions as they approach an ego-vehicle. The dataset comprises synchronized multi-modal data, including fisheye camera feeds, lidar laser scans, ultrasonic sensor readings, and motion capture-based 3D body poses, collected across diverse real-world scenarios. Key contributions include detailed annotations of 3D body joint positions synchronized with fisheye camera images, as well as accurate 3D pedestrian positions extracted from lidar data, facilitating robust benchmarking for perception algorithms. We release a portion of the dataset along with a comprehensive benchmark suite, featuring evaluation metrics for accuracy, efficiency, and scalability on embedded systems. By addressing real-world challenges such as sensor occlusions, dynamic environments, and hardware constraints, this dataset offers a unique resource for developing and evaluating state-of-the-art algorithms in pedestrian detection, 3D pose estimation and 4D trajectory and intention prediction. Additionally, we provide baseline performance metrics using custom neural network architectures and suggest future research directions to encourage the adoption and enhancement of the dataset. This work aims to serve as a foundation for researchers seeking to advance the capabilities of intelligent vehicles in near-field scenarios.

Valeo Near-Field: a novel dataset for pedestrian intent detection

TL;DR

The paper introduces the Valeo Near-Field dataset, a multimodal resource tailored for pedestrian intent detection around a stationary ego-vehicle. It combines synchronized fisheye camera streams, LiDAR, ultrasonic sensors, and motion-capture-based 3D poses to enable robust benchmarking of detection, 3D pose estimation, trajectory, and intent prediction in near-field scenarios. A 300-sequence corpus with a public-test subset and a dedicated benchmark suite provides end-to-end evaluation, including zone-based metrics and occlusion handling. This dataset addresses gaps in near-field perception under occlusion and multi-view conditions, with practical implications for enhancing safety in autonomous driving around pedestrians.

Abstract

This paper presents a novel dataset aimed at detecting pedestrians' intentions as they approach an ego-vehicle. The dataset comprises synchronized multi-modal data, including fisheye camera feeds, lidar laser scans, ultrasonic sensor readings, and motion capture-based 3D body poses, collected across diverse real-world scenarios. Key contributions include detailed annotations of 3D body joint positions synchronized with fisheye camera images, as well as accurate 3D pedestrian positions extracted from lidar data, facilitating robust benchmarking for perception algorithms. We release a portion of the dataset along with a comprehensive benchmark suite, featuring evaluation metrics for accuracy, efficiency, and scalability on embedded systems. By addressing real-world challenges such as sensor occlusions, dynamic environments, and hardware constraints, this dataset offers a unique resource for developing and evaluating state-of-the-art algorithms in pedestrian detection, 3D pose estimation and 4D trajectory and intention prediction. Additionally, we provide baseline performance metrics using custom neural network architectures and suggest future research directions to encourage the adoption and enhancement of the dataset. This work aims to serve as a foundation for researchers seeking to advance the capabilities of intelligent vehicles in near-field scenarios.
Paper Structure (14 sections, 5 figures, 3 tables)

This paper contains 14 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Applications
  4. Datasets
  5. The Valeo Near-Field Dataset
  6. Data Collection Setup
  7. Sensor Suite
  8. Synchronization of Multimodal Data
  9. Data Collection Process
  10. Experiments & Baseline Results
  11. Experiments
  12. Metrics
  13. Baseline Results
  14. Conclusion

Figures (5)

  • Figure 1: Valeo Near Field Dataset. Inference visualization on an exemplary scene. The four fisheye streams are shown on the left side, displaying the detected 2D body joints, 2D bounding boxes, and camera-consistent IDs for each pedestrian. On the right side, a rectangular representation of the vehicle is depicted along with the 3D pose estimations of both pedestrians, including head orientation vectors and upper-body orientation vectors. Additionally, a dotted circle highlights the 10m range around the vehicle.
  • Figure 2: Vehicle and LiDAR configuration.
  • Figure 3: Annotation Strategy. Various elements used to create 3D skeleton annotations are visible.
  • Figure 4: Vehicle and sensor configuration.
  • Figure 5: Annotation Visualization. Projection of 3D skeleton data on a 2D fisheye image.