Image-Based Visual Servoing for Enhanced Cooperation of Dual-Arm Manipulation

TL;DR

This work tackles pose synchronization in dual-arm cooperative manipulation under imperfect kinematics by introducing an image-based visual servoing (IBVS) framework that leverages inter-arm markers observed by wrist-mounted cameras. A customized interaction matrix that accounts for both arms' poses is derived, and the IBVS control is integrated with pose synchronization through a weighted cost $V= w_{d}V_{d}+w_{t}V_{t}+w_{s}V_{s}$, with stability proven via a Lyapunov argument. The approach is validated on real UR robots, showing reduced image-feature error and smaller force/torque fluctuations when the IBVS and synchronization terms are balanced ($w_{d}=w_{t}=w_{s}=0.5$), while extreme weightings can degrade performance at higher speeds. Overall, the method offers a robust, model-uncertainty-tolerant means to achieve fixture-free cooperative manipulation with rigorous stability guarantees and practical feasibility.

Abstract

The cooperation of a pair of robot manipulators is required to manipulate a target object without any fixtures. The conventional control methods coordinate the end-effector pose of each manipulator with that of the other using their kinematics and joint coordinate measurements. Yet, the manipulators' inaccurate kinematics and joint coordinate measurements can cause significant pose synchronization errors in practice. This paper thus proposes an image-based visual servoing approach for enhancing the cooperation of a dual-arm manipulation system. On top of the classical control, the visual servoing controller lets each manipulator use its carried camera to measure the image features of the other's marker and adapt its end-effector pose with the counterpart on the move. Because visual measurements are robust to kinematic errors, the proposed control can reduce the end-effector pose synchronization errors and the fluctuations of the interaction forces of the pair of manipulators on the move. Theoretical analyses have rigorously proven the stability of the closed-loop system. Comparative experiments on real robots have substantiated the effectiveness of the proposed control.

Figures (7)

• Figure 1: Two $6$-DOF serial manipulators serve as the left and the right arms in a dual-arm manipulation system. The target object that they manipulate is a cubic box. $\mathcal{F}_w$ and $\mathcal{F}_o$ denote the world frame and the object frame, respectively. $\mathcal{F}_{b\ast}$, $\mathcal{F}_{e\ast}$ and $\mathcal{F}_{c\ast}$ are the base, end-effector and camera frames of the left $\ast=l$ and the right $\ast=r$ arms. $\mathbf{T}_{o}$, $\mathbf{T}_{b\ast}$, $\mathbf{T}_{\ast}$, $\mathbf{T}^{b}_{\ast}$ and $\mathbf{T}^{\ast}_{o}$ indicate the transformations between the frames.
• Figure 2: The positions of the corner points $i=1,2,3,4$ expressed of the right arm's marker expressed in the end-effector frame $\mathcal{F}_{er}$ and in the left camera frame $\mathcal{F}_{cl}$ are $\mathbf{p}_{ri}$ and $\mathbf{m}_{li}$, respectively. $\mathbf{T}_{\ast}$ and $\mathbf{T}^{l}_{c}$ denote the transformations between the world frame $\mathcal{F}_{w}$, the end-effector frames $\mathcal{F}_{e\ast}$ and the camera frame $\mathcal{F}_{cl}$ with $\ast=l,r$.
• Figure 3: The experimental setup includes two Universal Robots UR3/UR3e manipulators. Both arms equip elastic tools on their flanges to transport a cubic box via frictional contacts. The camera mounted on the wrist of each arm captures the marker mounted on the wrist of the other arm.
• Figure 4: The Cartesian paths and the waypoints' orientations of the two arms' end effectors when the object moves along path1, path2 and path3, respectively. The final poses of both arms coincide with their initial poses.
• Figure 5: The errors of the image features $\mathbf{s}=(\mathbf{s}^\mathsf{T}_l,\mathbf{s}^\mathsf{T}_r)^\mathsf{T}$ from their desired (initial) values when transporting the object along path1 during the first group of experiments with $w_d=1.0$ and $w_t=1-w_s$.