Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Pallet Detection Using Synthetic Data

Henry Gann, Josiah Bull, Trevor Gee, Mahla Nejati

TL;DR

This paper tackles pallet detection in warehouse settings by exploiting synthetic data, including Unity-based scenes and domain randomisation, to train fast detectors like YOLOv8. A grid-search over hyper-parameters substantially boosts performance, yielding 69% and 50% gains in mAP50 for stacked and racked pallets, though individual pallets see a decline. The study also explores lighting robustness, a two-stage YOLOv8+SAM detector (which proves unstable), and domain randomisation via Isaac Sim, finding DR data can match Unity in effectiveness but may require careful data-volume and epoch management. The work highlights the potential of synthetic data to bridge the sim-to-real gap for complex pallet configurations, while outlining practical considerations for lighting, model selection, and data-generation strategies in real warehouses.

Abstract

The use of synthetic data in machine learning saves a significant amount of time when implementing an effective object detector. However, there is limited research in this domain. This study aims to improve upon previously applied implementations in the task of instance segmentation of pallets in a warehouse environment. This study proposes using synthetically generated domain-randomised data as well as data generated through Unity to achieve this. This study achieved performance improvements on the stacked and racked pallet categories by 69% and 50% mAP50, respectively when being evaluated on real data. Additionally, it was found that there was a considerable impact on the performance of a model when it was evaluated against images in a darker environment, dropping as low as 3% mAP50 when being evaluated on images with an 80% brightness reduction. This study also created a two-stage detector that used YOLOv8 and SAM, but this proved to have unstable performance. The use of domain-randomised data proved to have negligible performance improvements when compared to the Unity-generated data.

Improving Pallet Detection Using Synthetic Data

TL;DR

This paper tackles pallet detection in warehouse settings by exploiting synthetic data, including Unity-based scenes and domain randomisation, to train fast detectors like YOLOv8. A grid-search over hyper-parameters substantially boosts performance, yielding 69% and 50% gains in mAP50 for stacked and racked pallets, though individual pallets see a decline. The study also explores lighting robustness, a two-stage YOLOv8+SAM detector (which proves unstable), and domain randomisation via Isaac Sim, finding DR data can match Unity in effectiveness but may require careful data-volume and epoch management. The work highlights the potential of synthetic data to bridge the sim-to-real gap for complex pallet configurations, while outlining practical considerations for lighting, model selection, and data-generation strategies in real warehouses.

Abstract

The use of synthetic data in machine learning saves a significant amount of time when implementing an effective object detector. However, there is limited research in this domain. This study aims to improve upon previously applied implementations in the task of instance segmentation of pallets in a warehouse environment. This study proposes using synthetically generated domain-randomised data as well as data generated through Unity to achieve this. This study achieved performance improvements on the stacked and racked pallet categories by 69% and 50% mAP50, respectively when being evaluated on real data. Additionally, it was found that there was a considerable impact on the performance of a model when it was evaluated against images in a darker environment, dropping as low as 3% mAP50 when being evaluated on images with an 80% brightness reduction. This study also created a two-stage detector that used YOLOv8 and SAM, but this proved to have unstable performance. The use of domain-randomised data proved to have negligible performance improvements when compared to the Unity-generated data.
Paper Structure (23 sections, 11 figures, 2 tables)

This paper contains 23 sections, 11 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Synthetic Data
  4. Modern Deep Learning Techniques
  5. Pallet Detection
  6. Lighting within a Machine Learning Context
  7. Gaps and Challenges
  8. Methods
  9. Improving Model Performance
  10. Lighting
  11. YOLO + SAM
  12. Domain Randomisation
  13. Results
  14. Improving Model Performance
  15. Lighting
  16. ...and 8 more sections

Figures (11)

  • Figure 1: A synthetic image of a pallet rendered through Unity.
  • Figure 2: An example of domain randomisation of a pallet in a warehouse environment (Left) and a randomised environment with randomised colours (Right).
  • Figure 3: An example of 80% static brightness reduction on a real-world image
  • Figure 4: Overview of the YOLO + SAM two-stage detector.
  • Figure 5: Comparing the performance of our proposed method compared to the previous method by naidoo2023pallet.
  • ...and 6 more figures