Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects

Jakob Paul Zimmermann, Gerrit Holzbach, David Lerch

Abstract

Object detectors deployed in safety-critical environments can fail silently, e.g. missing pedestrians, workers, or other safety-critical objects without emitting any warning. Traditional Out Of Distribution (OOD) detection methods focus on identifying unfamiliar inputs, but do not directly predict functional failures of the detector itself. We introduce Knowledge Guided Failure Prediction (KGFP), a representation-based monitoring framework that treats missed safety-critical detections as anomalies to be detected at runtime. KGFP measures semantic misalignment between internal object detector features and visual foundation model embeddings using a dual-encoder architecture with an angular distance metric. A key property is that when either the detector is operating outside its competence or the visual foundation model itself encounters novel inputs, the two embeddings diverge, producing a high-angle signal that reliably flags unsafe images. We compare our novel KGFS method to baseline OOD detection methods. On COCO person detection, applying KGFP as a selective-prediction gate raises person recall among accepted images from 64.3% to 84.5% at 5% False Positive Rate (FPR), and maintains strong performance across six COCO-O visual domains, outperforming OOD baselines by large margins. Our code, models, and features are published at https://gitlab.cc-asp.fraunhofer.de/iosb_public/KGFP.

Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects

Abstract

Object detectors deployed in safety-critical environments can fail silently, e.g. missing pedestrians, workers, or other safety-critical objects without emitting any warning. Traditional Out Of Distribution (OOD) detection methods focus on identifying unfamiliar inputs, but do not directly predict functional failures of the detector itself. We introduce Knowledge Guided Failure Prediction (KGFP), a representation-based monitoring framework that treats missed safety-critical detections as anomalies to be detected at runtime. KGFP measures semantic misalignment between internal object detector features and visual foundation model embeddings using a dual-encoder architecture with an angular distance metric. A key property is that when either the detector is operating outside its competence or the visual foundation model itself encounters novel inputs, the two embeddings diverge, producing a high-angle signal that reliably flags unsafe images. We compare our novel KGFS method to baseline OOD detection methods. On COCO person detection, applying KGFP as a selective-prediction gate raises person recall among accepted images from 64.3% to 84.5% at 5% False Positive Rate (FPR), and maintains strong performance across six COCO-O visual domains, outperforming OOD baselines by large margins. Our code, models, and features are published at https://gitlab.cc-asp.fraunhofer.de/iosb_public/KGFP.

Paper Structure

This paper contains 30 sections, 4 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. ood Detection
  4. Runtime Monitoring for Safety
  5. Foundation Models for Robustness
  6. Method
  7. Problem Formulation
  8. Architecture: Dual-Encoder with Angular Metric
  9. Multi-Scale Feature Extraction
  10. Pre-Fusion Cross-Scale Attention
  11. Post-Fusion Transformer
  12. Dual Encoders
  13. Angular Failure Metric
  14. Training Objective
  15. Experimental Setup
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Overview of our proposed knowledge-guided failure prediction for object detectors. Traditional failure prediction methods including OOD utilize only internal features of the object detector. We leverage visual foundation model features to classify if an image is safe for person detection.
  • Figure 2: The dual-encoder architecture of our kgfp. We use pretrained DINO and YOLO models as backbones. For our kgfp we freeze the pretrained backbones and fine-tune a fusion framework. During evaluation the distance between DINO features and the fused features serves as the measure for our failure prediction.
  • Figure 3: KGFP performance on COCO Val. We plot person recall (blue) and precision (red) on the y-axis [in %] versus KGFP FPR on the x-axis [in %].