Adaptive Vision-Based Control of Redundant Robots with Null-Space Interaction for Human-Robot Collaboration

TL;DR

A new control scheme has been proposed for redundant robots in this paper, consisting of an adaptive vision-based control term in task space and an interactive controlterm in null space that allows the robot to autonomously carry out tasks in an unknown environment without prior calibration while also interacting with humans to deal with unforeseen changes under the redundant configuration.

Abstract

Human-robot collaboration aims to extend human ability through cooperation with robots. This technology is currently helping people with physical disabilities, has transformed the manufacturing process of companies, improved surgical performance, and will likely revolutionize the daily lives of everyone in the future. Being able to enhance the performance of both sides, such that human-robot collaboration outperforms a single robot/human, remains an open issue. For safer and more effective collaboration, a new control scheme has been proposed for redundant robots in this paper, consisting of an adaptive vision-based control term in task space and an interactive control term in null space. Such a formulation allows the robot to autonomously carry out tasks in an unknown environment without prior calibration while also interacting with humans to deal with unforeseen changes (e.g., potential collision, temporary needs) under the redundant configuration. The decoupling between task space and null space helps to explore the collaboration safely and effectively without affecting the main task of the robot end-effector. The stability of the closed-loop system has been rigorously proved with Lyapunov methods, and both the convergence of the position error in task space and that of the damping model in null space are guaranteed. The experimental results of a robot manipulator guided with the technology of augmented reality (AR) are presented to illustrate the performance of the control scheme.

Figures (9)

• Figure 1: Block diagram of the proposed control scheme. In the presence of an uncalibrated camera, the exact information about the image Jacobian matrix is unknown and denoted as $\hat{\bm J}_s(\bm r)$. In vision space, the robot end-effector is controlled to move to the desired position, that is, $\bm x\rightarrow\bm x_d$; In null space, the redundant joints are controlled to suit human intention under the desired damping model, that is, $c_d\dot{\bm q}=\bm d$, where $c_d$ is a positive constant and $\bm d$ denotes the control efforts from a human.
• Figure 2: The human can input via various interfaces, e.g., head-mounted display, haptic device, or tactile sensor, to shape the redundant configuration without affecting the robot end-effector.
• Figure 3: (a) Experimental setup of an AR-guided robot manipulator (b) A snapshot of the human-robot interface deployed on HoloLens 2. The sliders were placed on the left side, and the virtual manipulator model was on the right side, overlapped with the real manipulator. The human operator was pointing at joint 3 and giving a sign of moving up. The virtual manipulator model could also be placed somewhere else according to the situations.
• Figure 4: Snapshots of task 1. The manipulator was controlled to move to the desired point in vision space, while the operator exerted additional control efforts to make another human subject more comfortable via the AR interface. (a) $t = 0.0\space {\rm s}$: The end-effector started at the lower-right corner of the camera frame, and would move to the desired point at the center in $1.6\space {\rm s}$. (b) $t = 6.0 \space {\rm s}$: The operator began to drag sliders corresponding to joint 2 and joint 3 to change the manipulator's configuration because another human subject entered the workspace and stretched his body in an uncomfortable pose to fetch tools. (c) $t = 20.1 \space {\rm s}$: The manipulator adjusted its pose according to the command, while the position of the end-effector in vision space remained the same. Meanwhile, it enabled the human subject to have a safer and more comfortable workspace. (d) $t = 30.3 \space {\rm s}$: After the subject left, the robot's joint moved back to its original position according to the operator's command.
• Figure 5: Results of task 1: (a) top: The position errors of the end-effector in vision space; bottom: The human's control efforts applied to the joints. (b) The trajectory of the end-effector in vision space.

Theorems & Definitions (1)

• proof