Multi-Camera Hand-Eye Calibration for Human-Robot Collaboration in Industrial Robotic Workcells

TL;DR

This work tackles robust hand-eye calibration for multi-camera networks in industrial human-robot collaboration, addressing occlusions and large sensing setups. It introduces a non-linear optimization framework that enforces a shared board-to-end-effector transform and inter-camera poses, enabling simultaneous calibration of all cameras with respect to the robot base. By leveraging cross-detections and a reprojection-based objective, the method achieves superior accuracy with few images, validated on the METRIC dataset and real industrial workcells, and is released as open-source. The approach reduces downtime and improves monitoring reliability in complex workcells, marking a practical advance for camera-networked robot systems in industry.

Abstract

In industrial scenarios, effective human-robot collaboration relies on multi-camera systems to robustly monitor human operators despite the occlusions that typically show up in a robotic workcell. In this scenario, precise localization of the person in the robot coordinate system is essential, making the hand-eye calibration of the camera network critical. This process presents significant challenges when high calibration accuracy should be achieved in short time to minimize production downtime, and when dealing with extensive camera networks used for monitoring wide areas, such as industrial robotic workcells. Our paper introduces an innovative and robust multi-camera hand-eye calibration method, designed to optimize each camera's pose relative to both the robot's base and to each other camera. This optimization integrates two types of key constraints: i) a single board-to-end-effector transformation, and ii) the relative camera-to-camera transformations. We demonstrate the superior performance of our method through comprehensive experiments employing the METRIC dataset and real-world data collected on industrial scenarios, showing notable advancements over state-of-the-art techniques even using less than 10 images. Additionally, we release an open-source version of our multi-camera hand-eye calibration algorithm at https://github.com/davidea97/Multi-Camera-Hand-Eye-Calibration.git.

Figures (6)

• Figure 1: A large industrial robotic workcell equipped with a camera network around an ABB robot arm, enabling human-robot collaboration task in a carbon fiber draping process as foreseen in the DrapeBot European Project.
• Figure 2: Single-camera hand-eye calibration setup, used to derive the homogeneous transformations $AX=ZB$. Here, $A$ denotes the camera-to-board transformation and $B$ represents the pose of the robot's end-effector relative to its base $W$. While $X$ and $Z$ are the two unknown transformations.
• Figure 3: Industrial robotic workcell calibration; a checkerboard attached to the robot end-effector is moved to different positions in front of the surrounding sensors at about 4 meters away from the calibration pattern.
• Figure 4: Transformations chain in a single-camera hand-eye setup, illustrating the re-projection of calibration pattern corners onto the image plane. The robot's end effector pose relative to its base (W) is denoted by $T_{E}^{W}$, along with two unknown matrices: $X$, representing the hand-eye transformation, and $Z$, denoting the transformation between the board and the robot's end-effector pose.
• Figure 5: Multi-camera hand-eye setup, illustrating the geometric transformations optimized through the proposed multi-camera hand-eye calibration method. They include the single board-to-end-effector transformation common to all cameras, the spatial constraints among the cameras, and all the hand-eye transformations.