Automatic Robot Hand-Eye Calibration Enabled by Learning-Based 3D Vision

TL;DR

This work proposes a novel methodology that addresses the hand-eye calibration problem using the robot base as a reference, eliminating the need for external calibration objects or human intervention, and offers the potential to be completed in 1 second, which is the fastest compared to other 3D hand-eye calibration methods.

Abstract

Hand-eye calibration, as a fundamental task in vision-based robotic systems, aims to estimate the transformation matrix between the coordinate frame of the camera and the robot flange. Most approaches to hand-eye calibration rely on external markers or human assistance. We proposed Look at Robot Base Once (LRBO), a novel methodology that addresses the hand-eye calibration problem without external calibration objects or human support, but with the robot base. Using point clouds of the robot base, a transformation matrix from the coordinate frame of the camera to the robot base is established as I=AXB. To this end, we exploit learning-based 3D detection and registration algorithms to estimate the location and orientation of the robot base. The robustness and accuracy of the method are quantified by ground-truth-based evaluation, and the accuracy result is compared with other 3D vision-based calibration methods. To assess the feasibility of our methodology, we carried out experiments utilizing a low-cost structured light scanner across varying joint configurations and groups of experiments. The proposed hand-eye calibration method achieved a translation deviation of 0.930 mm and a rotation deviation of 0.265 degrees according to the experimental results. Additionally, the 3D reconstruction experiments demonstrated a rotation error of 0.994 degrees and a position error of 1.697 mm. Moreover, our method offers the potential to be completed in 1 second, which is the fastest compared to other 3D hand-eye calibration methods. Code is released at github.com/leihui6/LRBO.

Figures (17)

• Figure 1: Our proposed hand-eye calibration method. We estimate the transformation matrix between the robot base and camera frame using point clouds and one robot arm movement where the robot base as an object is detected and aligned with a 3D model.
• Figure 2: Our developed pipeline for the robot base 6D-Pose estimation.
• Figure 3: Overview of our proposed method. Given a point cloud ($\textbf{P}$) from a 3D camera, the robot base is detected and located as an RoI ($\textbf{P}^{'}$) via a learning-based 3D detection framework PV-RCNN++, which is trained and evaluated with our real-world dataset for the robot base task. In addition, the number of training datasets for robot base registration is increased by point cloud augmentation using a small number of aligned point clouds. We adopted a low-overlap designated learning-based framework, PREDATOR, to align the RoI with a 3D model ($\textbf{Q}$) of the robot base. The performance of the registration is also evaluated with our real data. Finally, the hand-eye calibration is solved with the registration result ($^{Cam}_{Base}\textbf{T}$) and the D-H parameter model ($^{Base}_{TCP}\textbf{T}$).
• Figure 4: 3D-IoU and an example. An example of 3D-IoU is give on the right where the 3D-IoU is about 0.78.
• Figure 5: An example of robot base detection where the robot base is shown in green.