Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AssemAI: Interpretable Image-Based Anomaly Detection for Manufacturing Pipelines

Renjith Prasad, Chathurangi Shyalika, Ramtin Zand, Fadi El Kalach, Revathy Venkataramanan, Ramy Harik, Amit Sheth

TL;DR

This paper introduces AssemAI, an interpretable image-based anomaly detection system tailored for smart manufacturing pipelines that leverages domain knowledge in data preparation, model development and reasoning, and implements several anomaly detection models on the derived image dataset.

Abstract

Anomaly detection in manufacturing pipelines remains a critical challenge, intensified by the complexity and variability of industrial environments. This paper introduces AssemAI, an interpretable image-based anomaly detection system tailored for smart manufacturing pipelines. Utilizing a curated image dataset from an industry-focused rocket assembly pipeline, we address the challenge of imbalanced image data and demonstrate the importance of image-based methods in anomaly detection. Our primary contributions include deriving an image dataset, fine-tuning an object detection model YOLO-FF, and implementing a custom anomaly detection model for assembly pipelines. The proposed approach leverages domain knowledge in data preparation, model development and reasoning. We implement several anomaly detection models on the derived image dataset, including a Convolutional Neural Network, Vision Transformer (ViT), and pre-trained versions of these models. Additionally, we incorporate explainability techniques at both user and model levels, utilizing ontology for user-level explanations and SCORE-CAM for in-depth feature and model analysis. Finally, the best-performing anomaly detection model and YOLO-FF are deployed in a real-time setting. Our results include ablation studies on the baselines and a comprehensive evaluation of the proposed system. This work highlights the broader impact of advanced image-based anomaly detection in enhancing the reliability and efficiency of smart manufacturing processes. The image dataset, codes to reproduce the results and additional experiments are available at https://github.com/renjithk4/AssemAI.

AssemAI: Interpretable Image-Based Anomaly Detection for Manufacturing Pipelines

TL;DR

This paper introduces AssemAI, an interpretable image-based anomaly detection system tailored for smart manufacturing pipelines that leverages domain knowledge in data preparation, model development and reasoning, and implements several anomaly detection models on the derived image dataset.

Abstract

Anomaly detection in manufacturing pipelines remains a critical challenge, intensified by the complexity and variability of industrial environments. This paper introduces AssemAI, an interpretable image-based anomaly detection system tailored for smart manufacturing pipelines. Utilizing a curated image dataset from an industry-focused rocket assembly pipeline, we address the challenge of imbalanced image data and demonstrate the importance of image-based methods in anomaly detection. Our primary contributions include deriving an image dataset, fine-tuning an object detection model YOLO-FF, and implementing a custom anomaly detection model for assembly pipelines. The proposed approach leverages domain knowledge in data preparation, model development and reasoning. We implement several anomaly detection models on the derived image dataset, including a Convolutional Neural Network, Vision Transformer (ViT), and pre-trained versions of these models. Additionally, we incorporate explainability techniques at both user and model levels, utilizing ontology for user-level explanations and SCORE-CAM for in-depth feature and model analysis. Finally, the best-performing anomaly detection model and YOLO-FF are deployed in a real-time setting. Our results include ablation studies on the baselines and a comprehensive evaluation of the proposed system. This work highlights the broader impact of advanced image-based anomaly detection in enhancing the reliability and efficiency of smart manufacturing processes. The image dataset, codes to reproduce the results and additional experiments are available at https://github.com/renjithk4/AssemAI.
Paper Structure (26 sections, 2 equations, 7 figures, 4 tables)

This paper contains 26 sections, 2 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Object Detection Models
  4. Zero-Shot Object Detection
  5. Anomaly detection
  6. Proposed Methodology
  7. Future Factories Multimodal Dataset
  8. Image filtering
  9. Image Cropping
  10. Text Guided Zero-Shot Object Detection
  11. Fine-tuning Object Detection Model with YOLOv9
  12. Problem Formulation
  13. Task Description
  14. Experiments
  15. Experiments with YOLO-FF Model
  16. ...and 11 more sections

Figures (7)

  • Figure 1: Overall Architecture of AssemAI. The figure illustrates the AssemAI pipeline, beginning with dataset preparation, which includes filtering and cropping images. Next, object detection is performed using zero-shot detection and a fine-tuned model (YOLO-FF). This is followed by anomaly detection using CNN, Custom-ViT, Pre-trained-ViT, and EfficientNet. The detection output is explained using SCORE-CAM for model-level explanations and process ontology for user-level explanations.
  • Figure 2: Structural Similarity between Normal and Anomalous Images
  • Figure 3: Score-CAM visualization shows the model mistakenly focusing on background elements, highlighted in red as the most important regions, rather than the assembly pipeline marked in blue. This emphasizes the need for cropping and object detection to improve accuracy by isolating the relevant parts of the image.
  • Figure 4: Experimental results with OWL-ViT. Class 1-5 denotes anomaly types: [No Anomaly], [NoNose], [NoNose,NoBody2], [NoNose,NoBody2,NoBody1] and [NoBody1] respectively
  • Figure 5: Verification of the explanations through process ontology
  • ...and 2 more figures