HEADS-UP: Head-Mounted Egocentric Dataset for Trajectory Prediction in Blind Assistance Systems

TL;DR

This paper introduces HEADS-UP, the first egocentric dataset collected from head-mounted cameras, designed specifically for trajectory prediction in blind assistance systems, and proposes a semi-local trajectory prediction approach to assess collision risks between blind individuals and pedestrians in dynamic environments.

Abstract

In this paper, we introduce HEADS-UP, the first egocentric dataset collected from head-mounted cameras, designed specifically for trajectory prediction in blind assistance systems. With the growing population of blind and visually impaired individuals, the need for intelligent assistive tools that provide real-time warnings about potential collisions with dynamic obstacles is becoming critical. These systems rely on algorithms capable of predicting the trajectories of moving objects, such as pedestrians, to issue timely hazard alerts. However, existing datasets fail to capture the necessary information from the perspective of a blind individual. To address this gap, HEADS-UP offers a novel dataset focused on trajectory prediction in this context. Leveraging this dataset, we propose a semi-local trajectory prediction approach to assess collision risks between blind individuals and pedestrians in dynamic environments. Unlike conventional methods that separately predict the trajectories of both the blind individual (ego agent) and pedestrians, our approach operates within a semi-local coordinate system, a rotated version of the camera's coordinate system, facilitating the prediction process. We validate our method on the HEADS-UP dataset and implement the proposed solution in ROS, performing real-time tests on an NVIDIA Jetson GPU through a user study. Results from both dataset evaluations and live tests demonstrate the robustness and efficiency of our approach.

Figures (4)

• Figure 1: With the increasing number of blind and visually impaired individuals, coupled with advancements in vision-based algorithms, there is a growing need for intelligent assistive tools that can inform the blind person in advance about potential collisions with dynamic obstacles, such as pedestrians. Unlike the traditional white cane, which is limited to detecting local collisions with static objects, these systems offer enhanced navigation safety by predicting and warning of dynamic threats.
• Figure 2: We use a ZED Mini stereo camera zedmini to capture the dataset, securely mounted on a cap using a custom 3D-printed attachment to ensure stable positioning during data collection.
• Figure 3: Example of RGB and depth images from each subset of the dataset. In the Easy subset, head movement is limited, while in the Hard subset, more drastic head movements are present. Both subsets have a controlled setup with a limited number of pedestrians. In contrast, the Uncontrolled setup features multiple pedestrians, a higher possibility of collisions, and drastic head movements. These diverse subsets enable the development and evaluation of algorithms in a variety of scenarios, facilitating research in blind navigation systems.
• Figure 4: An example of pedestrian trajectories before (red) and after (blue) applying smoothing techniques, demonstrating reduced noise and improved trajectory reliability. The X and Y coordinates are in meters.