Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness

TL;DR

The role of AI at the most relevant operational layers of AVs is analyzed, and the implications of the EU’s AI Act on AVs are discussed, highlighting the importance of the concept of a safety component.

Abstract

This study explores the complexities of integrating Artificial Intelligence (AI) into Autonomous Vehicles (AVs), examining the challenges introduced by AI components and the impact on testing procedures, focusing on some of the essential requirements for trustworthy AI. Topics addressed include the role of AI at various operational layers of AVs, the implications of the EU's AI Act on AVs, and the need for new testing methodologies for Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS). The study also provides a detailed analysis on the importance of cybersecurity audits, the need for explainability in AI decision-making processes and protocols for assessing the robustness and ethical behaviour of predictive systems in AVs. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology, highlighting the need for multidisciplinary expertise.

Figures (8)

• Figure 1: Key operational layers: localisation, perception, planning, control, human-vehicle interaction and system management. Adapted and expanded from jo2014development.
• Figure 2: Socially-aware AI for AVs. Perception, Prediction, and Planning, also referred to as the 3 Ps. Perception and planning augmented with prediction to safely yet effectively deploy AV that will interact with other social agents.
• Figure 3: Four levels of concreteness in AI component AV security. I, a vulnerability is known to exist for an AI. II, a vulnerability is shown for an AI that is potentially used in cars. III, the vulnerability has been shown on a testbed, or on a car in a safe environment (e.g., driver is aware of the test). IV, the exploit has been used on a car on the road.
• Figure 4: From commentary driving, requirements for explanations were gathered to inform the design of factual and counterfactual explanation algorithms. The algorithms receive input data from the different autonomous driving operations, provide a structured representation, and generate intelligible explanations to an end-user.
• Figure 5: Visualization of different transformation functions. The scene before transformation is followed by three different transformations.