Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review

TL;DR

This work presents the first comprehensive systematic literature review of explainable methods for safe and trustworthy AD and identifies five key contributions of XAI for safe and trustworthy AI in AD, which are interpretable design, interpretable surrogate models, interpretable monitoring, auxiliary explanations, and interpretable validation.

Abstract

Artificial Intelligence (AI) shows promising applications for the perception and planning tasks in autonomous driving (AD) due to its superior performance compared to conventional methods. However, inscrutable AI systems exacerbate the existing challenge of safety assurance of AD. One way to mitigate this challenge is to utilize explainable AI (XAI) techniques. To this end, we present the first comprehensive systematic literature review of explainable methods for safe and trustworthy AD. We begin by analyzing the requirements for AI in the context of AD, focusing on three key aspects: data, model, and agency. We find that XAI is fundamental to meeting these requirements. Based on this, we explain the sources of explanations in AI and describe a taxonomy of XAI. We then identify five key contributions of XAI for safe and trustworthy AI in AD, which are interpretable design, interpretable surrogate models, interpretable monitoring, auxiliary explanations, and interpretable validation. Finally, we propose a modular framework called SafeX to integrate these contributions, enabling explanation delivery to users while simultaneously ensuring the safety of AI models.

Figures (8)

• Figure 1: The structure of our survey. We present foundations into why we need trustworthy AI, an XAI taxonomy, and AD terminology. We then describe our survey methodology in detail so that our review is reproducible. In the analysis, we categorize existing XAI for AD approaches into five branches based on their different applications for AD: interpretable design, interpretable monitoring, interpretable surrogate models, auxiliary explanations and interpretable validation. A new conceptual framework is presented called SafeX based on our analysis. Finally, we discuss challenges and future directions.
• Figure 2: A taxonomy of XAI covering the most important concepts occurring in the literature of XAI for AD with terminology borrowed from Speith speith2022review.
• Figure 3: A typical system overview of autonomous driving systemspendleton2017perception. Arrows denote the flow of information. Orange boxes are hardware, grey boxes are software components. (V2V: vehicle-to-vehicle communication.)
• Figure 4: The query hierarchy used for the survey with a representative list of keywords corresponding to each query. Colours signal various depths of the search hierarchy.
• Figure 5: Overview of the review process as a flowchart. First, papers were retrieved according to the query hierarchy (blue box), then twice filtered by content (orange boxes). Numbers in parentheses show papers at the end of each step.

Theorems & Definitions (5)

• Definition 4.1: Interpretable By Design
• Definition 4.2: Interpretable Surrogate Models
• Definition 4.3: Interpretable Monitoring
• Definition 4.4: Auxiliary Explanations
• Definition 4.5: Interpretable Safety Validation