PEDESTRIAN: An Egocentric Vision Dataset for Obstacle Detection on Pavements

TL;DR

The paper tackles pedestrian sidewalk safety by introducing the PEDESTRIAN egocentric dataset, comprising 340 chest-height smartphone videos of 29 sidewalk obstacles and a balanced 14500-image subset for benchmarking. It evaluates 16 deep learning architectures across three obstacle taxonomy levels, demonstrating near-perfect obstacle-type accuracies and providing a reproducible benchmark framework. The dataset is publicly available (Zenodo) with accompanying code (GitHub), enabling researchers to advance obstacle detection for urban pedestrians. The work lays a foundation for robust, privacy-conscious obstacle detection systems that can inform safer pedestrian navigation in real-world urban environments.

Abstract

Walking has always been a primary mode of transportation and is recognized as an essential activity for maintaining good health. Despite the need for safe walking conditions in urban environments, sidewalks are frequently obstructed by various obstacles that hinder free pedestrian movement. Any object obstructing a pedestrian's path can pose a safety hazard. The advancement of pervasive computing and egocentric vision techniques offers the potential to design systems that can automatically detect such obstacles in real time, thereby enhancing pedestrian safety. The development of effective and efficient identification algorithms relies on the availability of comprehensive and well-balanced datasets of egocentric data. In this work, we introduce the PEDESTRIAN dataset, comprising egocentric data for 29 different obstacles commonly found on urban sidewalks. A total of 340 videos were collected using mobile phone cameras, capturing a pedestrian's point of view. Additionally, we present the results of a series of experiments that involved training several state-of-the-art deep learning algorithms using the proposed dataset, which can be used as a benchmark for obstacle detection and recognition tasks. The dataset can be used for training pavement obstacle detectors to enhance the safety of pedestrians in urban areas.

Figures (7)

• Figure 1: Example images for the 29 obstacle types in the PEDESTRIAN dataset.
• Figure 2: Pipeline for the generation of the PEDESTRIAN dataset: multiple videos of each obstacle type were taken using a smartphone camera from a pedestrian's point of view, and then individual frames were extracted from the collected videos.
• Figure 3: The taxonomy of obstacle types included in the dataset.
• Figure 4: Statistics of the dataset.
• Figure 5: Breakdown of the number of frames in the full PEDESTRIAN dataset for the three taxonomy levels: (a) Category, (b) Subcategory, and (c) Obstacle type; and, respectively, (d--f) for the balanced subset used in the benchmarking experiments.