Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Object detection characteristics in a learning factory environment using YOLOv8

Toni Schneidereit, Stefan Gohrenz, Michael Breuß

TL;DR

This study addresses industrial object detection in a learning factory by systematically varying object appearance across 15 workpieces and training 92 YOLOv8 models (in both nano and extra-large variants) to assess background and material influences on detection. It combines single-class and multi-class training with Layer-wise Relevance Propagation (LRP) heatmaps to probe explainability and potential overfitting, using $mAP$ and $IoU$-based metrics for evaluation. Key findings show that some materials (e.g., yellow/white plastics, light wood) are detected reliably, while others (aluminium, steel, black, red plastics) are challenging, and background/reflective cues can mislead predictions. The work provides a challenging, well-documented dataset and insights that inform dataset design, model explainability, and the development of robust industrial detectors for complex factory environments.

Abstract

AI-based object detection, and efforts to explain and investigate their characteristics, is a topic of high interest. The impact of, e.g., complex background structures with similar appearances as the objects of interest, on the detection accuracy and, beforehand, the necessary dataset composition are topics of ongoing research. In this paper, we present a systematic investigation of background influences and different features of the object to be detected. The latter includes various materials and surfaces, partially transparent and with shiny reflections in the context of an Industry 4.0 learning factory. Different YOLOv8 models have been trained for each of the materials on different sized datasets, where the appearance was the only changing parameter. In the end, similar characteristics tend to show different behaviours and sometimes unexpected results. While some background components tend to be detected, others with the same features are not part of the detection. Additionally, some more precise conclusions can be drawn from the results. Therefore, we contribute a challenging dataset with detailed investigations on 92 trained YOLO models, addressing some issues on the detection accuracy and possible overfitting.

Object detection characteristics in a learning factory environment using YOLOv8

TL;DR

This study addresses industrial object detection in a learning factory by systematically varying object appearance across 15 workpieces and training 92 YOLOv8 models (in both nano and extra-large variants) to assess background and material influences on detection. It combines single-class and multi-class training with Layer-wise Relevance Propagation (LRP) heatmaps to probe explainability and potential overfitting, using and -based metrics for evaluation. Key findings show that some materials (e.g., yellow/white plastics, light wood) are detected reliably, while others (aluminium, steel, black, red plastics) are challenging, and background/reflective cues can mislead predictions. The work provides a challenging, well-documented dataset and insights that inform dataset design, model explainability, and the development of robust industrial detectors for complex factory environments.

Abstract

AI-based object detection, and efforts to explain and investigate their characteristics, is a topic of high interest. The impact of, e.g., complex background structures with similar appearances as the objects of interest, on the detection accuracy and, beforehand, the necessary dataset composition are topics of ongoing research. In this paper, we present a systematic investigation of background influences and different features of the object to be detected. The latter includes various materials and surfaces, partially transparent and with shiny reflections in the context of an Industry 4.0 learning factory. Different YOLOv8 models have been trained for each of the materials on different sized datasets, where the appearance was the only changing parameter. In the end, similar characteristics tend to show different behaviours and sometimes unexpected results. While some background components tend to be detected, others with the same features are not part of the detection. Additionally, some more precise conclusions can be drawn from the results. Therefore, we contribute a challenging dataset with detailed investigations on 92 trained YOLO models, addressing some issues on the detection accuracy and possible overfitting.

Paper Structure

This paper contains 20 sections, 15 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Problem statement and contribution
  4. Background and dataset
  5. YOLOv8 and layer-wise relevance propagation
  6. Layer-wise relevance propagation and YOLOv8 heatmaps
  7. Performance measurement metrics
  8. Dataset creation and YOLO training
  9. Data collection
  10. Training and evaluation datasets
  11. Experiments and Results
  12. Single-class training and evaluation
  13. Training results
  14. Evaluation results
  15. Averaged results for training an evaluation
  16. ...and 5 more sections

Figures (15)

  • Figure 1: General overview of the Fischertechnik learning factory. Source: LFBilder
  • Figure 2: Overview of the 15 workpieces, from top to bottom: wood, metal, acrylic glass, PETG/plastic.
  • Figure 3: Similarities in the appearance of the (left) aluminium, (mid) steel, (right) grey workpieces.
  • Figure 4: Examples of certain workpieces showing similar characteristics as components of the learning factory.
  • Figure 5: Schematic representation of the YOLOv8 architecture with different modules and multi-scale detection. Source: so:sa:ra
  • ...and 10 more figures