Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

World Model Failure Classification and Anomaly Detection for Autonomous Inspection

Michelle Ho, Muhammad Fadhil Ginting, Isaac R. Ward, Andrzej Reinke, Mykel J. Kochenderfer, Ali-akbar Agha-Mohammadi, Shayegan Omidshafiei

TL;DR

A hybrid framework that combines supervised failure classification with anomaly detection, enabling classification of inspection tasks as a success, known failure, or anomaly case, and over 90% accuracy in distinguishing between successes, failures, and OOD cases is proposed.

Abstract

Autonomous inspection robots for monitoring industrial sites can reduce costs and risks associated with human-led inspection. However, accurate readings can be challenging due to occlusions, limited viewpoints, or unexpected environmental conditions. We propose a hybrid framework that combines supervised failure classification with anomaly detection, enabling classification of inspection tasks as a success, known failure, or anomaly (i.e., out-of-distribution) case. Our approach uses a world model backbone with compressed video inputs. This policy-agnostic, distribution-free framework determines classifications based on two decision functions set by conformal prediction (CP) thresholds before a human observer does. We evaluate the framework on gauge inspection feeds collected from office and industrial sites and demonstrate real-time deployment on a Boston Dynamics Spot. Experiments show over 90% accuracy in distinguishing between successes, failures, and OOD cases, with classifications occurring earlier than a human observer. These results highlight the potential for robust, anticipatory failure detection in autonomous inspection tasks or as a feedback signal for model training to assess and improve the quality of training data. Project website: https://autoinspection-classification.github.io

World Model Failure Classification and Anomaly Detection for Autonomous Inspection

TL;DR

A hybrid framework that combines supervised failure classification with anomaly detection, enabling classification of inspection tasks as a success, known failure, or anomaly case, and over 90% accuracy in distinguishing between successes, failures, and OOD cases is proposed.

Abstract

Autonomous inspection robots for monitoring industrial sites can reduce costs and risks associated with human-led inspection. However, accurate readings can be challenging due to occlusions, limited viewpoints, or unexpected environmental conditions. We propose a hybrid framework that combines supervised failure classification with anomaly detection, enabling classification of inspection tasks as a success, known failure, or anomaly (i.e., out-of-distribution) case. Our approach uses a world model backbone with compressed video inputs. This policy-agnostic, distribution-free framework determines classifications based on two decision functions set by conformal prediction (CP) thresholds before a human observer does. We evaluate the framework on gauge inspection feeds collected from office and industrial sites and demonstrate real-time deployment on a Boston Dynamics Spot. Experiments show over 90% accuracy in distinguishing between successes, failures, and OOD cases, with classifications occurring earlier than a human observer. These results highlight the potential for robust, anticipatory failure detection in autonomous inspection tasks or as a feedback signal for model training to assess and improve the quality of training data. Project website: https://autoinspection-classification.github.io
Paper Structure (23 sections, 4 equations, 7 figures, 5 tables)

This paper contains 23 sections, 4 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Out-of-Distribution Detection
  4. Failure Classification
  5. Hybrid Anomaly Detection and Failure Classification
  6. Conformal Prediction
  7. Methodology
  8. Problem Formulation
  9. Failure Classification and Anomaly Detection Framework
  10. Classifying Failures for Gauge Detection
  11. Dataset Creation
  12. Model Architecture and Training
  13. Conformal Prediction Threshold Calibration
  14. Calibration
  15. Evaluation
  16. ...and 8 more sections

Figures (7)

  • Figure 1: Our framework classifies successes, known failures, and anomalies with a world model backbone and conformal prediction thresholding, allowing for the robot to take informative actions based on the classification.
  • Figure 2: Decision function for failure classification.
  • Figure 3: Model pipeline: Input frame from video feed is optionally compressed, tokenized by Cosmos, propagated through a latent world model, decoded by Cosmos, optionally decompressed, and reconstructed into the next frame.
  • Figure 4: Examples of Gauge Image Classifications. We assume a fixed robot pose, so only the PTZ camera is controlled.
  • Figure 5: Threshold Calibration Phase
  • ...and 2 more figures