Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trustworthy Automated Driving through Qualitative Scene Understanding and Explanations

Nassim Belmecheri, Arnaud Gotlieb, Nadjib Lazaar, Helge Spieker

TL;DR

The paper addresses the challenge of trust and interpretability in automated driving by introducing the Qualitative Explainable Graph (QXG), a unified symbolic-qualitative scene representation constructed from sensor data to describe spatio-temporal relations in urban environments. It combines four qualitative calculi (QDC, RA, QTC_b, STAR_4) to capture distances, relative positions, and trajectories, enabling incremental real-time scene understanding and explainable reasoning. Action explanations are produced via one-class classifiers trained on QXG-derived relation chains, yielding interpretable justifications for vehicle actions with decision paths from tree-based models. Experiments on the nuScenes dataset demonstrate real-time QXG construction and effective action explanations with high precision/recall, supporting safer communication with passengers, VRUs, and external auditors, and outlining avenues for future V2V understanding and improved explanatory messaging.

Abstract

We present the Qualitative Explainable Graph (QXG): a unified symbolic and qualitative representation for scene understanding in urban mobility. QXG enables the interpretation of an automated vehicle's environment using sensor data and machine learning models. It leverages spatio-temporal graphs and qualitative constraints to extract scene semantics from raw sensor inputs, such as LiDAR and camera data, offering an intelligible scene model. Crucially, QXG can be incrementally constructed in real-time, making it a versatile tool for in-vehicle explanations and real-time decision-making across various sensor types. Our research showcases the transformative potential of QXG, particularly in the context of automated driving, where it elucidates decision rationales by linking the graph with vehicle actions. These explanations serve diverse purposes, from informing passengers and alerting vulnerable road users (VRUs) to enabling post-analysis of prior behaviours.

Trustworthy Automated Driving through Qualitative Scene Understanding and Explanations

TL;DR

The paper addresses the challenge of trust and interpretability in automated driving by introducing the Qualitative Explainable Graph (QXG), a unified symbolic-qualitative scene representation constructed from sensor data to describe spatio-temporal relations in urban environments. It combines four qualitative calculi (QDC, RA, QTC_b, STAR_4) to capture distances, relative positions, and trajectories, enabling incremental real-time scene understanding and explainable reasoning. Action explanations are produced via one-class classifiers trained on QXG-derived relation chains, yielding interpretable justifications for vehicle actions with decision paths from tree-based models. Experiments on the nuScenes dataset demonstrate real-time QXG construction and effective action explanations with high precision/recall, supporting safer communication with passengers, VRUs, and external auditors, and outlining avenues for future V2V understanding and improved explanatory messaging.

Abstract

We present the Qualitative Explainable Graph (QXG): a unified symbolic and qualitative representation for scene understanding in urban mobility. QXG enables the interpretation of an automated vehicle's environment using sensor data and machine learning models. It leverages spatio-temporal graphs and qualitative constraints to extract scene semantics from raw sensor inputs, such as LiDAR and camera data, offering an intelligible scene model. Crucially, QXG can be incrementally constructed in real-time, making it a versatile tool for in-vehicle explanations and real-time decision-making across various sensor types. Our research showcases the transformative potential of QXG, particularly in the context of automated driving, where it elucidates decision rationales by linking the graph with vehicle actions. These explanations serve diverse purposes, from informing passengers and alerting vulnerable road users (VRUs) to enabling post-analysis of prior behaviours.
Paper Structure (7 sections, 3 figures)

This paper contains 7 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Qualitative Calculi
  4. Qualitative Explainable Graph
  5. Action Explanation
  6. Experiments
  7. Conclusion

Figures (3)

  • Figure 1: Illustration of the successive construction process of the QXG over multiple frames. For simplification, only the rectangle algebra relation is depicted.
  • Figure 2: An overview of the explanation process and results for the trained action explanation classifiers.
  • Figure 3: Example action explanation overlaid on the LiDAR view: The car circled in red approaches the ego car, as captured by edge relations between these two objects above the images. Calculated from the specified calculi, the relations rationalise the stopping. NW: North west, NE: North east; Order of relations: $RA$, $QTC_b$, $QDC$, $STAR_4$.