Incorporating Explanations into Human-Machine Interfaces for Trust and Situation Awareness in Autonomous Vehicles

TL;DR

This paper tackles the opacity of ML-driven decisions in autonomous vehicles and the need for effective explanations to foster trust and hazard communication. It integrates explainable AI (XAI) and human–machine interfaces (HMI) under a 3W1H framework and proposes a unified situation-awareness model for inside and outside users, formalizing explanations as $I_e = (s, a, f, t)$ and aligning them with $Level_1$–$Level_2$ explanations. The authors validate the framework through a case study using interactive VQA-style explanations with the LLaVA system on driving scenes, followed by a user study assessing perceived safety and comfort and robustness to adversarial questions. Findings indicate that timely, faithful explanations improve user trust and safety, while adversarial or incorrect explanations can degrade them, underscoring the need for robust, inclusive HMI design.

Abstract

Autonomous vehicles often make complex decisions via machine learning-based predictive models applied to collected sensor data. While this combination of methods provides a foundation for real-time actions, self-driving behavior primarily remains opaque to end users. In this sense, explainability of real-time decisions is a crucial and natural requirement for building trust in autonomous vehicles. Moreover, as autonomous vehicles still cause serious traffic accidents for various reasons, timely conveyance of upcoming hazards to road users can help improve scene understanding and prevent potential risks. Hence, there is also a need to supply autonomous vehicles with user-friendly interfaces for effective human-machine teaming. Motivated by this problem, we study the role of explainable AI and human-machine interface jointly in building trust in vehicle autonomy. We first present a broad context of the explanatory human-machine systems with the "3W1H" (what, whom, when, how) approach. Based on these findings, we present a situation awareness framework for calibrating users' trust in self-driving behavior. Finally, we perform an experiment on our framework, conduct a user study on it, and validate the empirical findings with hypothesis testing.

Figures (5)

• Figure 1: An example of explanation communication to the pedestrians (top) and a human driver at the rear (down) by Waymo's self-driving car via its external HMI. The green bounding boxes have been manually added to indicate these signals. The figure drawn based on the content in waymo_message_display and waymo_driver2023.
• Figure 2: The proposed situation awareness framework for inside and outside users of an autonomous vehicle with XAI and representative HMIs
• Figure 3: Our experiment on the five chosen traffic scenes from the BDD-A dataset with the LLaVA multimodal transformer. While LLaVA seems to yield correct explanations on conventional questions (top) with actual actions (blue-colored text) + causal factors (green-colored text), it fails to generate factual explanations on the adversarial questions (bottom). The bounding boxes have manually been added to indicate causal factors inducing the chosen actions.
• Figure 4: Design of the case study based on the experiment in Figure \ref{['fig:LLaVA_exp']}. The participants judge the correctness of explanations for each of the five scenes presented. After getting experience with explanations, they are asked two more questions on their perceived safety and mental comfort with the role of explanations while using an autonomous vehicle. Users' responses are validated with a statistical significance test to draw a conclusion with the case study.
• Figure 5: The participants' responses to Question 11 and Question 13 in Fig \ref{['fig:user_study']} on their perceived feeling of safety and comfort with incorrect explanations