System-Level Safety Monitoring and Recovery for Perception Failures in Autonomous Vehicles

TL;DR

A Q-network called SPARQ is introduced that evaluates the safety of a plan generated by a planning algorithm, accounting for perception failures that the planning process may have over-looked and can be queried during system runtime to assess whether a proposed plan is safe for execution or poses potential safety risks.

Abstract

The safety-critical nature of autonomous vehicle (AV) operation necessitates development of task-relevant algorithms that can reason about safety at the system level and not just at the component level. To reason about the impact of a perception failure on the entire system performance, such task-relevant algorithms must contend with various challenges: complexity of AV stacks, high uncertainty in the operating environments, and the need for real-time performance. To overcome these challenges, in this work, we introduce a Q-network called SPARQ (abbreviation for Safety evaluation for Perception And Recovery Q-network) that evaluates the safety of a plan generated by a planning algorithm, accounting for perception failures that the planning process may have overlooked. This Q-network can be queried during system runtime to assess whether a proposed plan is safe for execution or poses potential safety risks. If a violation is detected, the network can then recommend a corrective plan while accounting for the perceptual failure. We validate our algorithm using the NuPlan-Vegas dataset, demonstrating its ability to handle cases where a perception failure compromises a proposed plan while the corrective plan remains safe. We observe an overall accuracy and recall of 90% while sustaining a frequency of 42Hz on the unseen testing dataset. We compare our performance to a popular reachability-based baseline and analyze some interesting properties of our approach in improving the safety properties of an AV pipeline.

Figures (7)

• Figure 1: We propose SPARQ, a run-time system-level safety monitor to detect and mitigate task-relevant perception failures. Here, the perception suite failed to detect the red car. This resulted in AV (the green car) planning an unsafe trajectory (blue line). SPARQ flags the planned trajectory as unsafe and suggests a corrective safe plan (cyan line) that accounts for the perception failure.
• Figure 2: Block diagram for task-relevant perception failure detection and recovery monitor. SPARQ learns a Q-network-based neural approximator for the last three blocks in this pipeline enabling it to operate at run-time at a frequency of $\sim$42 Hz.
• Figure 3: The SPARQ Network Structure
• Figure 4: Data Generation Structure. As our first step we sample a random position as our monitor input. The top row shows the case when an agent exists at the sampled position of the perception monitor (column (b)). We remove the agent from the ground truth (GT) scene to construct our perceived scene (column (d)). The bottom row shows the case when an agent does not exist. In such cases we use the GT scene as the perceived scene. In either case, our training sample comprises of (perceived scene, ego candidate plan), while the corresponding class label is obtained by evaluating the plan in the monitored scene (column (c)).
• Figure 5: Dataset Reward Distribution.

Theorems & Definitions (3)

• Remark 1
• Remark 2
• Remark 3