Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hazard-Aware Traffic Scene Graph Generation

Yaoqi Huang, Julie Stephany Berrio, Mao Shan, Stewart Worrall

TL;DR

A novel task, Traffic Scene Graph Generation, is introduced, which captures traffic-specific relations between prominent hazards and the ego vehicle and provides intuitive guidelines that stress prominent hazards by color-coding their severity and notating their effect mechanism and relative location to the ego vehicle.

Abstract

Maintaining situational awareness in complex driving scenarios is challenging. It requires continuously prioritizing attention among extensive scene entities and understanding how prominent hazards might affect the ego vehicle. While existing studies excel at detecting specific semantic categories and visually salient regions, they lack the ability to assess safety-relevance. Meanwhile, the generic spatial predicates either for foreground objects only or for all scene entities modeled by existing scene graphs are inadequate for driving scenarios. To bridge this gap, we introduce a novel task, Traffic Scene Graph Generation, which captures traffic-specific relations between prominent hazards and the ego vehicle. We propose a novel framework that explicitly uses traffic accident data and depth cues to supplement visual features and semantic information for reasoning. The output traffic scene graphs provide intuitive guidelines that stress prominent hazards by color-coding their severity and notating their effect mechanism and relative location to the ego vehicle. We create relational annotations on Cityscapes dataset and evaluate our model on 10 tasks from 5 perspectives. The results in comparative experiments and ablation studies demonstrate our capacity in ego-centric reasoning for hazard-aware traffic scene understanding.

Hazard-Aware Traffic Scene Graph Generation

TL;DR

A novel task, Traffic Scene Graph Generation, is introduced, which captures traffic-specific relations between prominent hazards and the ego vehicle and provides intuitive guidelines that stress prominent hazards by color-coding their severity and notating their effect mechanism and relative location to the ego vehicle.

Abstract

Maintaining situational awareness in complex driving scenarios is challenging. It requires continuously prioritizing attention among extensive scene entities and understanding how prominent hazards might affect the ego vehicle. While existing studies excel at detecting specific semantic categories and visually salient regions, they lack the ability to assess safety-relevance. Meanwhile, the generic spatial predicates either for foreground objects only or for all scene entities modeled by existing scene graphs are inadequate for driving scenarios. To bridge this gap, we introduce a novel task, Traffic Scene Graph Generation, which captures traffic-specific relations between prominent hazards and the ego vehicle. We propose a novel framework that explicitly uses traffic accident data and depth cues to supplement visual features and semantic information for reasoning. The output traffic scene graphs provide intuitive guidelines that stress prominent hazards by color-coding their severity and notating their effect mechanism and relative location to the ego vehicle. We create relational annotations on Cityscapes dataset and evaluate our model on 10 tasks from 5 perspectives. The results in comparative experiments and ablation studies demonstrate our capacity in ego-centric reasoning for hazard-aware traffic scene understanding.
Paper Structure (13 sections, 25 equations, 3 figures, 6 tables)

This paper contains 13 sections, 25 equations, 3 figures, 6 tables.

Table of Contents

  1. INTRODUCTION
  2. Related work
  3. Graph-based Traffic Scene Understanding
  4. Prior Knowledge and Knowledge Embedding
  5. methodology
  6. Ego-path Related Entities Selection
  7. Knowledge Graph Construction
  8. Knowledge Graph Embedding
  9. Traffic Scene Graph Generation
  10. Experiments and Results
  11. Tasks and Metrics
  12. Results and Discussions
  13. Conclusions

Figures (3)

  • Figure 1: Overview of our Hazard-Aware Traffic Scene Graph Generation ( HATS) model. The main scene graph branch (top) comprises three modules: 1) a Panoptic Segmentation (PS) Module for holistic perception of the surrounding environment, 2) an Ego-path Related Entities Selection (ERES) module that identifies and selects relevant candidate entities, and 3) a Traffic Scene Graph Generation (TSGG) module that computes ego-centric, traffic-specific relations among prominent hazards. The auxiliary knowledge branch (bottom) provides supplementary features and historical context to support entity representation and severity prediction within the TSGG module
  • Figure 2: Inference performance vs. training set size (5%–80% of total training set). For each size, five models were trained with five-fold splits, with 20% of training images held out for validation per fold.
  • Figure 3: Our ego-centric hazard-aware TSGs of traffic images