Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CableInspect-AD: An Expert-Annotated Anomaly Detection Dataset

Akshatha Arodi, Margaux Luck, Jean-Luc Bedwani, Aldo Zaimi, Ge Li, Nicolas Pouliot, Julien Beaudry, Gaétan Marceau Caron

TL;DR

CableInspect-AD tackles the gap in transferring visual anomaly detection to high-stakes industrial contexts by introducing a richly annotated, real-world dataset for robotic power-line cable inspection. The authors propose Enhanced-PatchCore, a threshold-estimation method that uses only training nominal images, and systematically evaluate it alongside open conversational Vision-Language Models in zero-shot settings through a robust cross-validation protocol. Results show strong zero-shot capabilities for VLMs but uneven performance across anomaly types and grades, while Enhanced-PatchCore remains competitive, especially for pixel-level segmentation and few-/many-shot settings. The dataset and evaluation framework establish a challenging, open benchmark to drive robust VAD in real-world industrial applications and encourage future work on zero-shot detection, localization, and segmentation in specialized domains.

Abstract

Machine learning models are increasingly being deployed in real-world contexts. However, systematic studies on their transferability to specific and critical applications are underrepresented in the research literature. An important example is visual anomaly detection (VAD) for robotic power line inspection. While existing VAD methods perform well in controlled environments, real-world scenarios present diverse and unexpected anomalies that current datasets fail to capture. To address this gap, we introduce $\textit{CableInspect-AD}$, a high-quality, publicly available dataset created and annotated by domain experts from Hydro-Québec, a Canadian public utility. This dataset includes high-resolution images with challenging real-world anomalies, covering defects with varying severity levels. To address the challenges of collecting diverse anomalous and nominal examples for setting a detection threshold, we propose an enhancement to the celebrated PatchCore algorithm. This enhancement enables its use in scenarios with limited labeled data. We also present a comprehensive evaluation protocol based on cross-validation to assess models' performances. We evaluate our $\textit{Enhanced-PatchCore}$ for few-shot and many-shot detection, and Vision-Language Models for zero-shot detection. While promising, these models struggle to detect all anomalies, highlighting the dataset's value as a challenging benchmark for the broader research community. Project page: https://mila-iqia.github.io/cableinspect-ad/.

CableInspect-AD: An Expert-Annotated Anomaly Detection Dataset

TL;DR

CableInspect-AD tackles the gap in transferring visual anomaly detection to high-stakes industrial contexts by introducing a richly annotated, real-world dataset for robotic power-line cable inspection. The authors propose Enhanced-PatchCore, a threshold-estimation method that uses only training nominal images, and systematically evaluate it alongside open conversational Vision-Language Models in zero-shot settings through a robust cross-validation protocol. Results show strong zero-shot capabilities for VLMs but uneven performance across anomaly types and grades, while Enhanced-PatchCore remains competitive, especially for pixel-level segmentation and few-/many-shot settings. The dataset and evaluation framework establish a challenging, open benchmark to drive robust VAD in real-world industrial applications and encourage future work on zero-shot detection, localization, and segmentation in specialized domains.

Abstract

Machine learning models are increasingly being deployed in real-world contexts. However, systematic studies on their transferability to specific and critical applications are underrepresented in the research literature. An important example is visual anomaly detection (VAD) for robotic power line inspection. While existing VAD methods perform well in controlled environments, real-world scenarios present diverse and unexpected anomalies that current datasets fail to capture. To address this gap, we introduce , a high-quality, publicly available dataset created and annotated by domain experts from Hydro-Québec, a Canadian public utility. This dataset includes high-resolution images with challenging real-world anomalies, covering defects with varying severity levels. To address the challenges of collecting diverse anomalous and nominal examples for setting a detection threshold, we propose an enhancement to the celebrated PatchCore algorithm. This enhancement enables its use in scenarios with limited labeled data. We also present a comprehensive evaluation protocol based on cross-validation to assess models' performances. We evaluate our for few-shot and many-shot detection, and Vision-Language Models for zero-shot detection. While promising, these models struggle to detect all anomalies, highlighting the dataset's value as a challenging benchmark for the broader research community. Project page: https://mila-iqia.github.io/cableinspect-ad/.
Paper Structure (33 sections, 3 equations, 26 figures, 3 tables)

This paper contains 33 sections, 3 equations, 26 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Datasets
  4. Anomaly detection algorithms
  5. CableInspect-AD dataset
  6. Creation and annotation
  7. Statistics
  8. Evaluation protocol
  9. Enhanced-PatchCore
  10. Experimental setting
  11. Results and discussion
  12. Performance variability in same category objects
  13. Enhanced-PatchCore - thresholding without a validation set
  14. Analysis of conversational VLMs
  15. Evaluating the impact of background removal
  16. ...and 18 more sections

Figures (26)

  • Figure 1: Examples of anomalies. On each image, the anomaly types (grades) are annotated (masks outlined). The grades here are (I)mportant, (L)ight, (C)omplete, (E)xtracted and (D)eep. Anomalies such as long scratches(I) are hard to spot, whereas deposit(I) and spaced strands(I) are easier.
  • Figure 2: Anomaly types and grades per cable. The grades are (I)mportant, (L)ight, (C)omplete, (E)xtracted, (P)artial, (D)eep and (S)uperficial. The anomalies are not distributed uniformly across all the cables.
  • Figure 3: The three cables have different numbers of images with varying anomaly ratios in the test set. The cables have 40, 46, and 30 folds, respectively. (a) shows the number of images in the test set over all the folds for each cable (x-axis), and (b) shows the ratio in the test set of the cables. Each point corresponds to the anomaly ratio in a fold. The identity line shows where a balanced dataset would be.
  • Figure 4: Image-level results of Enhanced-PatchCore (few-/many-shot) with the thresholding strategies and conversational VLMs (zero-shot). (a) and (b) show the mean and standard deviation over all folds for F1-score and FPR for the three cables. The x-axis indicates the number of images in the training set. (c) shows mean precision vs mean recall over all folds.
  • Figure 5: Image-level results in zero-shot setting using conversational VLMs and WinCLIP, and, few-shot and many-shot using Enhanced-PatchCore on CableInspect-AD_ cropped. Mean and standard deviation over all folds are reported for the three cables. On the figures, the x-axis indicates the number of images in the training set. (a) shows F1-score. For Enhanced-PatchCore, the metrics are computed using different thresholding strategies. (b) AUROC for Enhanced-PatchCore and WinCLIP.
  • ...and 21 more figures