Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unsupervised Welding Defect Detection Using Audio And Video

Georg Stemmer, Jose A. Lopez, Juan A. Del Hoyo Ontiveros, Arvind Raju, Tara Thimmanaik, Sovan Biswas

TL;DR

This work tackles real-time weld defect detection in robotic arc welding using unsupervised anomaly detection on audio and video signals. It introduces a large, real-world multimodal dataset and analyzes acoustic, visual, and fused approaches, showing that late fusion can achieve an average AUC around $0.92$ across 11 defect types. The audio and video pipelines use lightweight architectures (1D CNN auto-encoder for audio; SlowFast feature extraction with an auto-encoder for video), achieving fast detection latencies and robust performance for several defect classes. The findings demonstrate the practicality of multimodal, real-time defect detection in manufacturing and point to future directions like joint multimodal models and deployment in real production with labeled defects for validation.

Abstract

In this work we explore the application of AI to robotic welding. Robotic welding is a widely used technology in many industries, but robots currently do not have the capability to detect welding defects which get introduced due to various reasons in the welding process. We describe how deep-learning methods can be applied to detect weld defects in real-time by recording the welding process with microphones and a camera. Our findings are based on a large database with more than 4000 welding samples we collected which covers different weld types, materials and various defect categories. All deep learning models are trained in an unsupervised fashion because the space of possible defects is large and the defects in our data may contain biases. We demonstrate that a reliable real-time detection of most categories of weld defects is feasible both from audio and video, with improvements achieved by combining both modalities. Specifically, the multi-modal approach achieves an average Area-under-ROC-Curve (AUC) of 0.92 over all eleven defect types in our data. We conclude the paper with an analysis of the results by defect type and a discussion of future work.

Unsupervised Welding Defect Detection Using Audio And Video

TL;DR

This work tackles real-time weld defect detection in robotic arc welding using unsupervised anomaly detection on audio and video signals. It introduces a large, real-world multimodal dataset and analyzes acoustic, visual, and fused approaches, showing that late fusion can achieve an average AUC around across 11 defect types. The audio and video pipelines use lightweight architectures (1D CNN auto-encoder for audio; SlowFast feature extraction with an auto-encoder for video), achieving fast detection latencies and robust performance for several defect classes. The findings demonstrate the practicality of multimodal, real-time defect detection in manufacturing and point to future directions like joint multimodal models and deployment in real production with labeled defects for validation.

Abstract

In this work we explore the application of AI to robotic welding. Robotic welding is a widely used technology in many industries, but robots currently do not have the capability to detect welding defects which get introduced due to various reasons in the welding process. We describe how deep-learning methods can be applied to detect weld defects in real-time by recording the welding process with microphones and a camera. Our findings are based on a large database with more than 4000 welding samples we collected which covers different weld types, materials and various defect categories. All deep learning models are trained in an unsupervised fashion because the space of possible defects is large and the defects in our data may contain biases. We demonstrate that a reliable real-time detection of most categories of weld defects is feasible both from audio and video, with improvements achieved by combining both modalities. Specifically, the multi-modal approach achieves an average Area-under-ROC-Curve (AUC) of 0.92 over all eleven defect types in our data. We conclude the paper with an analysis of the results by defect type and a discussion of future work.
Paper Structure (15 sections, 2 equations, 3 figures, 25 tables)

This paper contains 15 sections, 2 equations, 3 figures, 25 tables.

Table of Contents

  1. Introduction
  2. Data Set
  3. Recording Setup, Data Collection Procedure, and Limitations
  4. Welding Sample Distribution
  5. Experimental Setup
  6. Acoustic Anomaly Detection
  7. Visual Anomaly Detection
  8. Multi-modal Anomaly Detection
  9. Results
  10. Acoustic Anomaly Detection
  11. Visual Anomaly Detection
  12. Multi-modal Anomaly Detection
  13. Conclusions and Future Work
  14. Sample Distributions
  15. Sample Images

Figures (3)

  • Figure 1: KML camera and computer.
  • Figure 2: Accept-reject probability of multi-modal predictions on test data.
  • Figure 3: Welding Samples