Revolutionizing Validation and Verification: Explainable Testing Methodologies for Intelligent Automotive Decision-Making Systems

TL;DR

The paper addresses the challenge of validating autonomous driving decision making where DM models are opaque and V&V is resource intensive. It proposes an explainable testing methodology that combines a test oracle, explanation generator, and an interactive test chatbot with an LLM driven explainable test-scenario generator and a simulation-based validation environment. The approach is structured into four phases—requirements gathering, scenario generation, V&V execution, and explainability evaluation—and complemented by a community repository for datasets and benchmarks. If effective, this framework could reduce diagnostic time, improve transparency, and strengthen stakeholder trust in ADS deployments.

Abstract

Autonomous Driving Systems (ADS) use complex decision-making (DM) models with multimodal sensory inputs, making rigorous validation and verification (V&V) essential for safety and reliability. These models pose challenges in diagnosing failures, tracing anomalies, and maintaining transparency, with current manual testing methods being inefficient and labor-intensive. This vision paper presents a methodology that integrates explainability, transparency, and interpretability into V&V processes. We propose refining V&V requirements through literature reviews and stakeholder input, generating explainable test scenarios via large language models (LLMs), and enabling real-time validation in simulation environments. Our framework includes test oracle, explanation generation, and a test chatbot, with empirical studies planned to evaluate improvements in diagnostic efficiency and transparency. Our goal is to streamline V&V, reduce resources, and build user trust in autonomous technologies.

Figures (1)

• Figure 1: The proposed empirical methodology overview.