Explaining Autonomous Vehicles with Intention-aware Policy Graphs

TL;DR

The paper addresses the challenge of explainability in autonomous driving by introducing post-hoc, model-agnostic teleological explanations based on Intention-aware Policy Graphs that represent an ego vehicle's behavior through desires and intentions. The approach builds global and local explanations from nuScenes data, enabling assessments of legal compliance, safety, and potential dataset vulnerabilities, while supporting interactive questions about intended goals and planned actions. Key contributions include a framework for integrating desires and intentions, a discrete state-action representation with interpretable predicates, and intention metrics that quantify interpretability and reliability, plus demonstrations of scene-level analyses and exposure of biases under varying visibility. The work advances trust and accountability in AVs by providing human-centric explanations and actionable insights for regulators, developers, and the public, with future directions toward user studies and multi-agent coordination.

Abstract

The potential to improve road safety, reduce human driving error, and promote environmental sustainability have enabled the field of autonomous driving to progress rapidly over recent decades. The performance of autonomous vehicles has significantly improved thanks to advancements in Artificial Intelligence, particularly Deep Learning. Nevertheless, the opacity of their decision-making, rooted in the use of accurate yet complex AI models, has created barriers to their societal trust and regulatory acceptance, raising the need for explainability. We propose a post-hoc, model-agnostic solution to provide teleological explanations for the behaviour of an autonomous vehicle in urban environments. Building on Intention-aware Policy Graphs, our approach enables the extraction of interpretable and reliable explanations of vehicle behaviour in the nuScenes dataset from global and local perspectives. We demonstrate the potential of these explanations to assess whether the vehicle operates within acceptable legal boundaries and to identify possible vulnerabilities in autonomous driving datasets and models.

Figures (4)

• Figure 1: Intention metrics for safe desires with $C = 0.5$. Globally, we can attribute an intention to fulfil any safe desire to the vehicle’s behaviour $75.4\%$ of the time, with $91.6\%$ certainty that the associated desires will be fulfilled (first metrics from the right).
• Figure 2: Intention metrics for unsafe desires with $C = 0.5$. Globally, we can attribute an intention to fulfil any unsafe desire to the vehicle’s behaviour $6.2\%$ of the time, with $85.9\%$ certainty that associated desires will be fulfilled (first metrics from the right).
• Figure 3: Abstract representation of scene 1084 of nuScenes. The ego vehicle is represented in red, with its final state marked with a red dot. Non-ego vehicles are represented in grey. The scene is captured at state $s = 20$, when the ego vehicle is nearly stationary in reaction to the abrupt lane intrusion of another vehicle.
• Figure 4: Temporal evolution of the ego vehicle's intention at each state in scene 1084, for different desires. For interpretability purposes, only desires that attain an intention value greater than 0.2 at least once throughout the scene are included in the visualisation. Vertical lines in the plot refer to the fulfilment of desires of the corresponding colour.