CC-SGG: Corner Case Scenario Generation using Learned Scene Graphs

TL;DR

CC-SGG introduces Corner Case Scenario Generation using Learned Scene Graphs, converting regular driving scenes into corner-case graphs with heterogeneous graph neural networks. The pipeline builds scene graphs from simulation data, encodes them with HGNNs that incorporate edge attributes via attention and triple embeddings, and perturbs graphs to produce corner-case graphs whose edges guide scenario generation in the CARLA simulator. Evaluations on a CARLA/open-scenario–based dataset show a prediction accuracy of $89.9\%$ and an ROC-AUC of $0.96$, with generated scenarios exposing safety weaknesses across five baseline autonomous driving methods. The work demonstrates a scalable, graph-based path to augmenting corner-case coverage for AV safety testing, paving the way for broader robustness analyses.

Abstract

Corner case scenarios are an essential tool for testing and validating the safety of autonomous vehicles (AVs). As these scenarios are often insufficiently present in naturalistic driving datasets, augmenting the data with synthetic corner cases greatly enhances the safe operation of AVs in unique situations. However, the generation of synthetic, yet realistic, corner cases poses a significant challenge. In this work, we introduce a novel approach based on Heterogeneous Graph Neural Networks (HGNNs) to transform regular driving scenarios into corner cases. To achieve this, we first generate concise representations of regular driving scenes as scene graphs, minimally manipulating their structure and properties. Our model then learns to perturb those graphs to generate corner cases using attention and triple embeddings. The input and perturbed graphs are then imported back into the simulation to generate corner case scenarios. Our model successfully learned to produce corner cases from input scene graphs, achieving 89.9% prediction accuracy on our testing dataset. We further validate the generated scenarios on baseline autonomous driving methods, demonstrating our model's ability to effectively create critical situations for the baselines.

Figures (5)

• Figure 1: Overview of our proposed methodology. We first extract semantics and relationships from simulated scenarios to generate heterogeneous scene graphs following a pre-defined generic knowledge graph ontology and grammar. Our model then encodes graph attributes generating concise feature representations to predict links corresponding to corner case scene graphs. The output of the model is a perturbed scene graph of an input regular driving graph. Scene graphs of the predicted corner case and the input regular driving graph are then used to generate corner case scenarios in simulation.
• Figure 2: Simulated scenario from regular driving (up) and its associated scene graph (down) of a bicycle in front of the ego vehicle travelling in the opposite lane. Self-edge attributes correspond to location $(x=133, y=133)$, speed $(v_x=20 m/s, v_y=0 m/s)$, and braking status. Cross-edge attributes correspond to the spatial relations between actors.
• Figure 3: Classification evaluation curves over a range of thresholds generated from the model's predictions in the testing dataset. The thresholds correspond to the optimum binary classification thresholds based on the ROC curve.
• Figure 4: Ground truth (up) and predicted corner case scene graph (down) generated from the regular scenario in Fig. \ref{['fig:fig2']}. The cyclist is now in the same lane as the ego vehicle and at an unsafe distance. The model has mostly successfully predicted the corner case edges but has generated two corner case scenes in the same graph.
• Figure 5: Ablation study to find the optimal architecture that minimises the validation loss and increases the F1-score in our testing dataset.