Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prediction of Occluded Pedestrians in Road Scenes using Human-like Reasoning: Insights from the OccluRoads Dataset

Melo Castillo Angie Nataly, Martin Serrano Sergio, Salinas Carlota, Sotelo Miguel Angel

TL;DR

This work addresses predicting occluded pedestrians in road scenes, where traditional detectors underperform relative to human perception. It introduces the OccluRoads dataset, containing real and virtual road scenes with rich occlusion labels, and a knowledge-driven predictor that combines a knowledge graph, knowledge graph embeddings, and Bayesian inference to reason about occlusion. The approach yields an F1 score of 0.91, substantially outperforming image-based baselines by leveraging contextual cues such as zebra crossings and vehicle states. The results demonstrate the practical value of integrating knowledge reasoning and synthetic data to improve safety-critical pedestrian predictions in autonomous driving and motivate future dataset expansion and benchmark development.

Abstract

Pedestrian detection is a critical task in autonomous driving, aimed at enhancing safety and reducing risks on the road. Over recent years, significant advancements have been made in improving detection performance. However, these achievements still fall short of human perception, particularly in cases involving occluded pedestrians, especially entirely invisible ones. In this work, we present the Occlusion-Rich Road Scenes with Pedestrians (OccluRoads) dataset, which features a diverse collection of road scenes with partially and fully occluded pedestrians in both real and virtual environments. All scenes are meticulously labeled and enriched with contextual information that encapsulates human perception in such scenarios. Using this dataset, we developed a pipeline to predict the presence of occluded pedestrians, leveraging Knowledge Graph (KG), Knowledge Graph Embedding (KGE), and a Bayesian inference process. Our approach achieves a F1 score of 0.91, representing an improvement of up to 42% compared to traditional machine learning models.

Prediction of Occluded Pedestrians in Road Scenes using Human-like Reasoning: Insights from the OccluRoads Dataset

TL;DR

This work addresses predicting occluded pedestrians in road scenes, where traditional detectors underperform relative to human perception. It introduces the OccluRoads dataset, containing real and virtual road scenes with rich occlusion labels, and a knowledge-driven predictor that combines a knowledge graph, knowledge graph embeddings, and Bayesian inference to reason about occlusion. The approach yields an F1 score of 0.91, substantially outperforming image-based baselines by leveraging contextual cues such as zebra crossings and vehicle states. The results demonstrate the practical value of integrating knowledge reasoning and synthetic data to improve safety-critical pedestrian predictions in autonomous driving and motivate future dataset expansion and benchmark development.

Abstract

Pedestrian detection is a critical task in autonomous driving, aimed at enhancing safety and reducing risks on the road. Over recent years, significant advancements have been made in improving detection performance. However, these achievements still fall short of human perception, particularly in cases involving occluded pedestrians, especially entirely invisible ones. In this work, we present the Occlusion-Rich Road Scenes with Pedestrians (OccluRoads) dataset, which features a diverse collection of road scenes with partially and fully occluded pedestrians in both real and virtual environments. All scenes are meticulously labeled and enriched with contextual information that encapsulates human perception in such scenarios. Using this dataset, we developed a pipeline to predict the presence of occluded pedestrians, leveraging Knowledge Graph (KG), Knowledge Graph Embedding (KGE), and a Bayesian inference process. Our approach achieves a F1 score of 0.91, representing an improvement of up to 42% compared to traditional machine learning models.

Paper Structure

This paper contains 19 sections, 1 equation, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Pedestrian occlusion datasets
  4. Pedestrian occlusion prediction
  5. Methodology
  6. Dataset Collection
  7. Data Labeling
  8. Experiments
  9. Model architecture
  10. KG Generation
  11. KGE Learning
  12. Bayesian inference and Prediction
  13. Implementation
  14. Experimental setup
  15. Results and Discussions
  16. ...and 4 more sections

Figures (6)

  • Figure 1: OccluRoads dataset examples
  • Figure 2: Dataset Annotations
  • Figure 3: Annotations to represent the contextual information of a road scene
  • Figure 4: Knowledge-based pipeline
  • Figure 5: Occluded Pedestrian Ontology
  • ...and 1 more figures