Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Incorporating System-level Safety Requirements in Perception Models via Reinforcement Learning

Weisi Fan, Jesse Lane, Qisai Liu, Soumik Sarkar, Tichakorn Wongpiromsarn

TL;DR

This work addresses the gap where standard perception losses neglect downstream safety in autonomous systems. It proposes a reinforcement learning framework that injects system-level safety objectives by translating rulebook specifications into safety scores used as rewards to fine-tune probabilistic perception models. A hybrid reward combines IoU-based perception performance with rule-based safety violations, enabling training in non-differentiable simulations like CARLA and integrating depth with pixel outputs from a PIX2SEQ detector. Experimental results show improved alignment between perception and safety objectives, better focus on prioritized, safety-relevant objects, and robust performance under varied weather conditions, demonstrating the practical potential of safety-aware perception training.

Abstract

Perception components in autonomous systems are often developed and optimized independently of downstream decision-making and control components, relying on established performance metrics like accuracy, precision, and recall. Traditional loss functions, such as cross-entropy loss and negative log-likelihood, focus on reducing misclassification errors but fail to consider their impact on system-level safety, overlooking the varying severities of system-level failures caused by these errors. To address this limitation, we propose a novel training paradigm that augments the perception component with an understanding of system-level safety objectives. Central to our approach is the translation of system-level safety requirements, formally specified using the rulebook formalism, into safety scores. These scores are then incorporated into the reward function of a reinforcement learning framework for fine-tuning perception models with system-level safety objectives. Simulation results demonstrate that models trained with this approach outperform baseline perception models in terms of system-level safety.

Incorporating System-level Safety Requirements in Perception Models via Reinforcement Learning

TL;DR

This work addresses the gap where standard perception losses neglect downstream safety in autonomous systems. It proposes a reinforcement learning framework that injects system-level safety objectives by translating rulebook specifications into safety scores used as rewards to fine-tune probabilistic perception models. A hybrid reward combines IoU-based perception performance with rule-based safety violations, enabling training in non-differentiable simulations like CARLA and integrating depth with pixel outputs from a PIX2SEQ detector. Experimental results show improved alignment between perception and safety objectives, better focus on prioritized, safety-relevant objects, and robust performance under varied weather conditions, demonstrating the practical potential of safety-aware perception training.

Abstract

Perception components in autonomous systems are often developed and optimized independently of downstream decision-making and control components, relying on established performance metrics like accuracy, precision, and recall. Traditional loss functions, such as cross-entropy loss and negative log-likelihood, focus on reducing misclassification errors but fail to consider their impact on system-level safety, overlooking the varying severities of system-level failures caused by these errors. To address this limitation, we propose a novel training paradigm that augments the perception component with an understanding of system-level safety objectives. Central to our approach is the translation of system-level safety requirements, formally specified using the rulebook formalism, into safety scores. These scores are then incorporated into the reward function of a reinforcement learning framework for fine-tuning perception models with system-level safety objectives. Simulation results demonstrate that models trained with this approach outperform baseline perception models in terms of system-level safety.

Paper Structure

This paper contains 23 sections, 4 theorems, 9 equations, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Preliminaries
  3. Reinforcement Learning Basics
  4. System-Level Safety Specifications
  5. Proposed Framework
  6. Components
  7. Perception Component
  8. Control Component
  9. Simulator
  10. Rulebook Component
  11. Reinforcement Learning Formulation
  12. Policy Gradient Algorithm
  13. Implementation Details
  14. Perception Component
  15. Simulator
  16. ...and 8 more sections

Key Result

Lemma 1

Suppose (1) $\mathit{rb}_1(x_t) = \mathit{rb}_2(x_t) = 0$ at arbitrary time $t$, (2) the set $O$ of objects in the same lane and in front of the ego vehicle remains the same at time $t$ and $t + \Delta t$, and (3) $a_{brake,i}$ is the maximum braking rate of object $o_i$. Then, applying $CTRL$ at ti

Figures (4)

  • Figure 1: Overview of the proposed feedback learning framework that incorporates both traditional vision-based metrics and safety metrics derived from a rulebook to train or fine-tune an object detection model in the perception component of an autonomous system; the object detection model is reframed as a reinforcement learning agent where, input images are considered as state, bounding box and class outputs are considered as action, and a combination of object detection and safety metrics is considered as the reward function.
  • Figure 2: Detection result comparison among pre-trained and fine-tuned model
  • Figure 3: Training Losses of different settings
  • Figure 4: Different weather condition simulation of same scenario

Theorems & Definitions (5)

  • Remark 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Corollary 1