Learning-Based Vision Systems for Semi-Autonomous Forklift Operation in Industrial Warehouse Environments

TL;DR

This work addresses the challenge of low-cost, reliable pallet and pallet-hole perception for semi-autonomous forklifts using a monocular camera. It evaluates two YOLO-based detectors, YOLOv8 and YOLOv11, with Optuna-driven hyperparameter tuning and two post-processing strategies (centroid and IoU) to map holes to pallets for precise fork alignment. The study demonstrates that a three-model YOLOv8 setup achieves strong generalization on pallets, while YOLOv11, especially when finely tuned, delivers stable convergence and improved localization; together, they demonstrate the feasibility of retrofittable, scale-friendly vision-perception modules for warehouse automation. The proposed pallet-hole mapping component provides actionable spatial representations that enable accurate navigation and fork insertion, offering a practical path toward safer, more economical, and intelligent warehouse logistics.

Abstract

The automation of material handling in warehouses increasingly relies on robust, low cost perception systems for forklifts and Automated Guided Vehicles (AGVs). This work presents a vision based framework for pallet and pallet hole detection and mapping using a single standard camera. We utilized YOLOv8 and YOLOv11 architectures, enhanced through Optuna driven hyperparameter optimization and spatial post processing. An innovative pallet hole mapping module converts the detections into actionable spatial representations, enabling accurate pallet and pallet hole association for forklift operation. Experiments on a custom dataset augmented with real warehouse imagery show that YOLOv8 achieves high pallet and pallet hole detection accuracy, while YOLOv11, particularly under optimized configurations, offers superior precision and stable convergence. The results demonstrate the feasibility of a cost effective, retrofittable visual perception module for forklifts. This study proposes a scalable approach to advancing warehouse automation, promoting safer, economical, and intelligent logistics operations.

Figures (3)

• Figure 1: Annotated images from the dataset.
• Figure 2: Pipeline for pallet and pallet-hole detection and association.
• Figure 3: Spatial association techniques to link pallet holes with respective pallets.