System-Level Analysis of Module Uncertainty Quantification in the Autonomy Pipeline

TL;DR

The paper tackles the ambiguity of uncertainty in learning-enabled autonomous systems by proposing two system-level analyses: (1) generating module-level uncertainty specifications through assume-guarantee (quotient) contracts, and (2) quantifying system robustness via input perturbations using sub-level set estimation. The methods are demonstrated on two real-world systems—the Autonomous Driving System and a Runway Incursion Detection System—yielding insights into how upstream uncertainty measures interact with downstream decision-makers and how design choices trade off performance and robustness. A data-driven, open-loop framework is emphasized, with detailed procedures for probabilistic specification estimation and GP-based robustness assessment. The work advances practical methodologies for robust-by-design autonomy, while outlining limitations and avenues for extending to closed-loop settings and broader validation.

Abstract

Modern autonomous systems with machine learning components often use uncertainty quantification to help produce assurances about system operation. However, there is a lack of consensus in the community on what uncertainty is and how to perform uncertainty quantification. In this work, we propose that uncertainty measures should be understood within the context of overall system design and operation. To this end, we present two novel analysis techniques. First, we produce a probabilistic specification on a module's uncertainty measure given a system specification. Second, we propose a method to measure a system's input-output robustness in order to compare system designs and quantify the impact of making a system uncertainty-aware. In addition to this theoretical work, we present the application of these analyses on two real-world autonomous systems: an autonomous driving system and an aircraft runway incursion detection system. We show that our analyses can determine desired relationships between module uncertainty and error, provide visualizations of how well an uncertainty measure is being used by a system, produce principled comparisons between different uncertainty measures and decision-making algorithm designs, and provide insights into system vulnerabilities and tradeoffs.

Figures (9)

• Figure 1: (a) An example of a sequential, modularized autonomy pipeline, in which $x$ is the state, $u$ is the control input, and $C(u)$ represents a cost function. Module $i$ has output $y_i$ with associated uncertainty $\sigma_i$. (b) Analysis 1: We propose that generating a specification on a module's uncertainty measure will help us to apply our system-level perspective. Using assume-guarantee contract theory, we treat this as a problem of finding the contract of an upstream module of interest (Contract 1) given the system contract and the contract of the downstream decision-making modules (Contract 2). We show some example results in which we use data to generate histograms that help to characterize the Contract 2. (c) Analysis 2: We propose to measure a system's input-output robustness by searching for all input errors such that the system output incurs an evaluation cost below a threshold. In the example results shown, we display functions and level sets that are the solution to the search problem. Different colors and line styles indicate different system designs. For both analyses, the light green color indicates evaluation on the Runway Incursion Detection system and the dark green color indicates evaluation on the Autonomous Driving system.
• Figure 2: The Autonomous Driving system, with an example scene from the nuScenes dataset. The ego agent must navigate to its goal in the presence of other agents, of which only the nearest agent is shown.
• Figure 3: The Runway Incursion Detection system, with an example image that might be input to the detector on the aircraft. From this aerial view of the runway, the aircraft must determine if there exist ground vehicles which could cause a runway incursion. Red circles correspond to detections. Yellow circles correspond to "probationary" tracks that are not fully initiated yet. Blue circles correspond to fully initiated tracks. The numbers correspond to detection confidence scores at that instant in time.
• Figure 4: (Running Example) (Top) A plot showing the raw data used to estimate $P(C(u) > c | \sigma_1, \mathbf{1}(e_1 < e_{thres}), z)$ for when the prediction error is below the threshold and the ego agent and target agent are close together. (Bottom) Plots showing one of the scenarios in the dataset with the associated plan for three different settings of the uncertainty value. Since this is the agent-avoiding planner, we indeed see that as the uncertainty value increases, the plan (dark blue solid line) takes a more conservative maneuver to reach the goal. The evaluation cost here is the safety cost.
• Figure 5: (Running Example) For the agent-avoiding planner, we show the final $P(C(u) > c | \sigma_1, \mathbf{1}(e_1 < e_{thres}), z)$ values for (a) evaluation using the safety cost and (b) evaluation using the holistic cost. For each plot, we set the values of $\mathbf{1}(e_1 < e_{thres})$ and $z$ and show how the probability values vary by the uncertainty value. The value of $e_{thres}$ is 1.5, the value of $c$ is 0.02, and a low ego-agent distance is defined as a distance less than 5 meters, while a high ego-agent distance is a distance greater than 5 meters. In general, these values can be chosen arbitrarily by the system designer. As the uncertainty value increases, the agent-avoiding planner keeps the ego agent further away from other agents, which can make the ego safer but worse at holistic driving.