Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning-based Inverse Perception Contracts and Applications

Dawei Sun, Benjamin C. Yang, Sayan Mitra

TL;DR

This work tackles safety-critical control under perception uncertainty by learning inverse perception contracts (IPCs) that map a perceived value $\hat{y}$ and system state $x$ to an ellipsoid $\mathcal{E}(c_\theta(x,\hat{y}), C_\theta(x,\hat{y}))$ containing the ground-truth $y$ with high probability. IPCs are trained as neural-network $c_\theta$ and $C_\theta$ heads using empirical risk with a hinge loss and a volume-penalizing regularizer, with probabilistic correctness guaranteed by a PAC-style bound. The approach is validated on a vision-based quadcopter landing task, where the IPC achieves an empirical containment error as low as $0.19\%$ and enables a robust landing strategy that succeeds where a baseline fails. The results demonstrate a practical pathway to incorporate quantified perception uncertainty into safety-critical controllers for real-world autonomous systems.

Abstract

Perception modules are integral in many modern autonomous systems, but their accuracy can be subject to the vagaries of the environment. In this paper, we propose a learning-based approach that can automatically characterize the error of a perception module from data and use this for safe control. The proposed approach constructs an inverse perception contract (IPC) which generates a set that contains the ground-truth value that is being estimated by the perception module, with high probability. We apply the proposed approach to study a vision pipeline deployed on a quadcopter. With the proposed approach, we successfully constructed an IPC for the vision pipeline. We then designed a control algorithm that utilizes the learned IPC, with the goal of landing the quadcopter safely on a landing pad. Experiments show that with the learned IPC, the control algorithm safely landed the quadcopter despite the error from the perception module, while the baseline algorithm without using the learned IPC failed to do so.

Learning-based Inverse Perception Contracts and Applications

TL;DR

This work tackles safety-critical control under perception uncertainty by learning inverse perception contracts (IPCs) that map a perceived value and system state to an ellipsoid containing the ground-truth with high probability. IPCs are trained as neural-network and heads using empirical risk with a hinge loss and a volume-penalizing regularizer, with probabilistic correctness guaranteed by a PAC-style bound. The approach is validated on a vision-based quadcopter landing task, where the IPC achieves an empirical containment error as low as and enables a robust landing strategy that succeeds where a baseline fails. The results demonstrate a practical pathway to incorporate quantified perception uncertainty into safety-critical controllers for real-world autonomous systems.

Abstract

Perception modules are integral in many modern autonomous systems, but their accuracy can be subject to the vagaries of the environment. In this paper, we propose a learning-based approach that can automatically characterize the error of a perception module from data and use this for safe control. The proposed approach constructs an inverse perception contract (IPC) which generates a set that contains the ground-truth value that is being estimated by the perception module, with high probability. We apply the proposed approach to study a vision pipeline deployed on a quadcopter. With the proposed approach, we successfully constructed an IPC for the vision pipeline. We then designed a control algorithm that utilizes the learned IPC, with the goal of landing the quadcopter safely on a landing pad. Experiments show that with the learned IPC, the control algorithm safely landed the quadcopter despite the error from the perception module, while the baseline algorithm without using the learned IPC failed to do so.
Paper Structure (24 sections, 2 theorems, 9 equations, 5 figures, 1 table)

This paper contains 24 sections, 2 theorems, 9 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. Perception-based control
  4. Analysis of systems with black-box modules
  5. Learning inverse perception contracts
  6. Inverse perception contracts
  7. Learning inverse perception contracts from data
  8. Probabilistic correctness of the inverse perception contract
  9. Application study: safe quadcopter landing
  10. The safe landing problem
  11. The quadcopter
  12. The workspace
  13. Coordinate frames
  14. The vision-based perception module
  15. The task specification
  16. ...and 9 more sections

Key Result

Theorem 1

For any $\epsilon>0$, and a random training set $S$ with $N$ i.i.d. samples, with probability at least $1-2 \exp(-2N\epsilon^2)$, the following inequality holds, where $p$ is the number of scalar parameters of the neural network, and $\tilde{\ell}(\cdot) = \min\{1, \ell(\cdot)\}$ is the truncated hinge loss, and $L_g$ is the Lipschitz constant of $g_{\theta}$ w.r.t. $\theta$.

Figures (5)

  • Figure 1: An autonomous system that entails a perception module and the proposed inverse perception contract.
  • Figure 2: The quadcopter and the landing pad.
  • Figure 3: The state machine of the safe landing algorithm.
  • Figure 4: Illustration of the safe landing algorithm. The green ellipsoids are the output of the IPC. (a) The uncertainty of the measurement is above the threshold. In this case, the quadcopter will follow the green line and navigate to a new waypoint (blue). (b) The uncertainty of the measurement is below the threshold. In this case, the quadcopter first moves horizontally and then descends to complete the landing. (c) The quadcopter safely lands on the landing pad.
  • Figure 5: 2D visualization of 10 runs. Each run is visualized in a unique color. The crosses mark the positions where the baseline approach turns off the motors. The stars mark the locations where the approach using the learned IPC turns off the motors. Circles correspond to the measurements made by the proposed approach. The center of the circle is the location where measurement is made, and size of the circle corresponds to the size of the ellipsoid computed by IPC.

Theorems & Definitions (4)

  • Definition 1: Error of an inverse perception contract
  • Theorem 1
  • Remark
  • Proposition 1