Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synthetic Dataset Generation and Learning From Demonstration Applied to Industrial Manipulation

Alireza Barekatain, Hamed Rahimi Nohooji, Holger Voos

TL;DR

The paper tackles the challenge of flexible industrial manipulation by combining a synthetic dataset generation pipeline for robust pose estimation with Learning-from-Demonstration to replace manual robot programming. It builds a photorealistic, shader-enhanced synthetic data workflow in BlenderProc to train pose-estimation models (demonstrated with PVNet) on industrial parts, including symmetry-aware viewpoint selection. It also deploys a ROS-based LfD framework that uses kinesthetic teaching and Dynamic Movement Primitives to learn motion sequences, enabling robots to adapt trajectories from new starting configurations. Collectively, the work enables non-expert operators to rapidly introduce new tasks and parts without expert programming, supporting mass customization and flexible production lines.

Abstract

The aim of this study is to investigate an automated industrial manipulation pipeline, where assembly tasks can be flexibly adapted to production without the need for a robotic expert, both for the vision system and the robot program. The objective of this study is first, to develop a synthetic-dataset-generation pipeline with a special focus on industrial parts, and second, to use Learning-from-Demonstration (LfD) methods to replace manual robot programming, so that a non-robotic expert/process engineer can introduce a new manipulation task by teaching it to the robot.

Synthetic Dataset Generation and Learning From Demonstration Applied to Industrial Manipulation

TL;DR

The paper tackles the challenge of flexible industrial manipulation by combining a synthetic dataset generation pipeline for robust pose estimation with Learning-from-Demonstration to replace manual robot programming. It builds a photorealistic, shader-enhanced synthetic data workflow in BlenderProc to train pose-estimation models (demonstrated with PVNet) on industrial parts, including symmetry-aware viewpoint selection. It also deploys a ROS-based LfD framework that uses kinesthetic teaching and Dynamic Movement Primitives to learn motion sequences, enabling robots to adapt trajectories from new starting configurations. Collectively, the work enables non-expert operators to rapidly introduce new tasks and parts without expert programming, supporting mass customization and flexible production lines.

Abstract

The aim of this study is to investigate an automated industrial manipulation pipeline, where assembly tasks can be flexibly adapted to production without the need for a robotic expert, both for the vision system and the robot program. The objective of this study is first, to develop a synthetic-dataset-generation pipeline with a special focus on industrial parts, and second, to use Learning-from-Demonstration (LfD) methods to replace manual robot programming, so that a non-robotic expert/process engineer can introduce a new manipulation task by teaching it to the robot.
Paper Structure (3 sections, 2 equations, 4 figures)

This paper contains 3 sections, 2 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Pose Estimation
  3. Learning from Demonstration

Figures (4)

  • Figure 1: The overview of the automated industrial manipulation pipeline.
  • Figure 2: An example of synthetic dataset generation. (a) and (b) show the input to the process, where the CAD file is used to recreate the geometry and shaders are customized based on the real appearance and material of the object, resulting in (c).
  • Figure 3: Samples from the validation set of a trained PVNet. The green bounding box is the ground-truth pose and the blue bounding box is the estimated pose.
  • Figure 4: The overall workflow of teaching a pick-and-place task. The human trains the method by guiding the robot through the desired movement (dotted trajectory). LfD planner creates a new trajectory from the received starting point to the desired end point (dash-dotted trajectory)