Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LucidGrasp: Robotic Framework for Autonomous Manipulation of Laboratory Equipment with Different Degrees of Transparency via 6D Pose Estimation

Maria Makarova, Daria Trinitatova, Qian Liu, Dzmitry Tsetserukou

TL;DR

The proposed robotic framework can be applied for laboratory automation, since it allows solving the problem of performing non-trivial manipulation tasks with the analysis of object poses of varying degrees of transparency and liquid levels, requiring high accuracy and repeatability.

Abstract

Many modern robotic systems operate autonomously, however they often lack the ability to accurately analyze the environment and adapt to changing external conditions, while teleoperation systems often require special operator skills. In the field of laboratory automation, the number of automated processes is growing, however such systems are usually developed to perform specific tasks. In addition, many of the objects used in this field are transparent, making it difficult to analyze them using visual channels. The contributions of this work include the development of a robotic framework with autonomous mode for manipulating liquid-filled objects with different degrees of transparency in complex pose combinations. The conducted experiments demonstrated the robustness of the designed visual perception system to accurately estimate object poses for autonomous manipulation, and confirmed the performance of the algorithms in dexterous operations such as liquid dispensing. The proposed robotic framework can be applied for laboratory automation, since it allows solving the problem of performing non-trivial manipulation tasks with the analysis of object poses of varying degrees of transparency and liquid levels, requiring high accuracy and repeatability.

LucidGrasp: Robotic Framework for Autonomous Manipulation of Laboratory Equipment with Different Degrees of Transparency via 6D Pose Estimation

TL;DR

The proposed robotic framework can be applied for laboratory automation, since it allows solving the problem of performing non-trivial manipulation tasks with the analysis of object poses of varying degrees of transparency and liquid levels, requiring high accuracy and repeatability.

Abstract

Many modern robotic systems operate autonomously, however they often lack the ability to accurately analyze the environment and adapt to changing external conditions, while teleoperation systems often require special operator skills. In the field of laboratory automation, the number of automated processes is growing, however such systems are usually developed to perform specific tasks. In addition, many of the objects used in this field are transparent, making it difficult to analyze them using visual channels. The contributions of this work include the development of a robotic framework with autonomous mode for manipulating liquid-filled objects with different degrees of transparency in complex pose combinations. The conducted experiments demonstrated the robustness of the designed visual perception system to accurately estimate object poses for autonomous manipulation, and confirmed the performance of the algorithms in dexterous operations such as liquid dispensing. The proposed robotic framework can be applied for laboratory automation, since it allows solving the problem of performing non-trivial manipulation tasks with the analysis of object poses of varying degrees of transparency and liquid levels, requiring high accuracy and repeatability.

Paper Structure

This paper contains 18 sections, 8 figures.

Table of Contents

  1. INTRODUCTION
  2. FRAMEWORK OVERVIEW
  3. System Architecture
  4. Visual Perception Module
  5. 6D Pose Estimation
  6. Predicting the Liquid Shape and Vessel Neck
  7. Decision-Making and Execution Module
  8. Simulated Environment
  9. Trajectory Planning Submodule
  10. Trajectory Transfer Submodule
  11. EXPERIMENTS
  12. Defining Work Area for 6D-pose Estimation
  13. Changing camera height at a fixed distance
  14. Changing Camera Distance with a Fixed Height
  15. 6D-pose Estimation for Complex Object Combinations
  16. ...and 3 more sections

Figures (8)

  • Figure 1: Overview of the proposed robotic framework. The outputs of the main modules are highlighted in yellow, ROS servers in orange.
  • Figure 2: Dataset objects (left) and object segmentation masks (right).
  • Figure 3: Example of predicting the neck of a vessel and the shape of the liquid inside the vessel in an occluded environment.
  • Figure 4: Position and rotation errors of target objects during pose estimation: 1 -- flask, 2 -- glass beaker, 3 -- graduated cylinder, 4 -- pipette, 5 -- test tube, 6 -- 6-hole tube rack, 7 -- 25-hole tube rack.
  • Figure 5: Dependence of position and rotation errors, averaged over all objects, on the distance of the camera to the objects.
  • ...and 3 more figures