Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Trustworthy Automated Driving through Qualitative Scene Understanding and Explanations

Nassim Belmecheri, Arnaud Gotlieb, Nadjib Lazaar, Helge Spieker

TL;DR

The Qualitative Explainable Graph (QXG), which is a unified symbolic and qualitative representation for scene understanding in urban mobility, is introduced, which enables interpreting an automated vehicle's environment using sensor data and machine learning models.

Abstract

Understanding driving scenes and communicating automated vehicle decisions are key requirements for trustworthy automated driving. In this article, we introduce the Qualitative Explainable Graph (QXG), which is a unified symbolic and qualitative representation for scene understanding in urban mobility. The QXG enables interpreting an automated vehicle's environment using sensor data and machine learning models. It utilizes spatio-temporal graphs and qualitative constraints to extract scene semantics from raw sensor inputs, such as LiDAR and camera data, offering an interpretable scene model. A QXG can be incrementally constructed in real-time, making it a versatile tool for in-vehicle explanations across various sensor types. Our research showcases the potential of QXG, particularly in the context of automated driving, where it can rationalize decisions by linking the graph with observed actions. These explanations can serve diverse purposes, from informing passengers and alerting vulnerable road users to enabling post-hoc analysis of prior behaviors.

Towards Trustworthy Automated Driving through Qualitative Scene Understanding and Explanations

TL;DR

The Qualitative Explainable Graph (QXG), which is a unified symbolic and qualitative representation for scene understanding in urban mobility, is introduced, which enables interpreting an automated vehicle's environment using sensor data and machine learning models.

Abstract

Understanding driving scenes and communicating automated vehicle decisions are key requirements for trustworthy automated driving. In this article, we introduce the Qualitative Explainable Graph (QXG), which is a unified symbolic and qualitative representation for scene understanding in urban mobility. The QXG enables interpreting an automated vehicle's environment using sensor data and machine learning models. It utilizes spatio-temporal graphs and qualitative constraints to extract scene semantics from raw sensor inputs, such as LiDAR and camera data, offering an interpretable scene model. A QXG can be incrementally constructed in real-time, making it a versatile tool for in-vehicle explanations across various sensor types. Our research showcases the potential of QXG, particularly in the context of automated driving, where it can rationalize decisions by linking the graph with observed actions. These explanations can serve diverse purposes, from informing passengers and alerting vulnerable road users to enabling post-hoc analysis of prior behaviors.
Paper Structure (22 sections, 7 figures, 4 tables, 1 algorithm)

This paper contains 22 sections, 7 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Qualitative Calculi.
  4. Qualitative Scene Understanding.
  5. Qualitative Explainable Graph
  6. Qualitative Explainable Graph Builder
  7. Bridging the Gap: Connecting Actions and QXG
  8. Action Description
  9. Action Extraction
  10. Explaining Actions through QXG
  11. Experimental Evaluation
  12. Experimental Setup
  13. (RQ1) QXG Construction
  14. (RQ2) One-Class vs. Multi-Class Classification Performance
  15. Evaluation of One-Class Classifiers
  16. ...and 7 more sections

Figures (7)

  • Figure 1: Illustration of the incremental construction of the QXG over three frames. For the sake of simplicity, only the rectangle algebra (RA) relation is depicted on the edges.
  • Figure 2: QXG Action Explanation Process: Previous relations extracted from the action-annotated QXG are classified based on the target action.
  • Figure 3: Construction time of QXG per frame
  • Figure 4: Scenario of a pedestrian passing a cross-walk and causing stopping actions in the approaching vehicles.
  • Figure 5: Scenario of two pedestrians walking close to the roadway and causing stopping actions in the approaching blue vehicle while the red vehicle overtakes it
  • ...and 2 more figures