Experimental Study on Automatically Assembling Custom Catering Packages With a 3-DOF Delta Robot Using Deep Learning Methods

TL;DR

The paper addresses automatic catering package packing using a 3-DOF Delta parallel robot by introducing a perception-to-action pipeline that fuses detection (YOLOv5) and segmentation (FastSAM) with a geometry-based grasp strategy. It presents the Catering Packages Objects (CPO) dataset for Persian-made items, demonstrates real-time transmission of perception outputs to the robot, and reports substantial autonomous packing performance. Key contributions include the CPO dataset (1400 images, 19 classes, 4000 annotations), a robust detection-segmentation-geometry workflow, and a calibrated robot-vision interface achieving around 85% packing success in tests. This work advances practical packaging automation in hygienic settings and provides a foundation for future enhancements using graph neural networks and advanced detectors such as YOLOv9.

Abstract

This paper introduces a pioneering experimental study on the automated packing of a catering package using a two-fingered gripper affixed to a 3-degree-of-freedom Delta parallel robot. A distinctive contribution lies in the application of a deep learning approach to tackle this challenge. A custom dataset, comprising 1,500 images, is meticulously curated for this endeavor, representing a noteworthy initiative as the first dataset focusing on Persian-manufactured products. The study employs the YOLOV5 model for object detection, followed by segmentation using the FastSAM model. Subsequently, rotation angle calculation is facilitated with segmentation masks, and a rotated rectangle encapsulating the object is generated. This rectangle forms the basis for calculating two grasp points using a novel geometrical approach involving eigenvectors. An extensive experimental study validates the proposed model, where all pertinent information is seamlessly transmitted to the 3-DOF Delta parallel robot. The proposed algorithm ensures real-time detection, calibration, and the fully autonomous packing process of a catering package, boasting an impressive over 80\% success rate in automatic grasping. This study marks a significant stride in advancing the capabilities of robotic systems for practical applications in packaging automation.

Figures (9)

• Figure 1: This figure demonstrates how two images get imported to the later proposed network, one is an image of the intended pack and the other objects under the robot. After detection, segmentation and geometrical calculations, the DPR and the gripper take action to pack the required objects according to the input image.
• Figure 2: Wrong detections of YOLOV5 pre-tuned model.
• Figure 3: SAM vs FastSAM segmentation output using everything mode.
• Figure 4: MinAreaRect rotation angle($\gamma$) algorithm explained and converting it into actual rotation angle($\psi$).
• Figure 5: Propsed geometrical method for obtaining grasp points using object's eigenvector.