Hybrid Controller for Robot Manipulators in Task-Space with Visual-Inertial Feedback

TL;DR

The paper tackles robust task-space trajectory tracking for robot manipulators under kinematic/dynamic uncertainties by fusing visual and inertial measurements. It develops a hybrid observer on $SE_2(3)$ to estimate end-effector pose from an IMU and stereo vision, and couples this with a hybrid, potential-based controller on $SE(3)$ that switches to ensure global asymptotic stability. The authors prove stability results and validate the method via simulations on a 6-DOF hydraulic manipulator, showing improved tracking without explicit inverse kinematics. This vision‑aided, hybrid approach reduces sensitivity to kinematic calibration errors and enhances task-space performance in challenging environments.

Abstract

This paper presents a visual-inertial-based control strategy to address the task space control problem of robot manipulators. To this end, an observer-based hybrid controller is employed to control end-effector motion. In addition, a hybrid observer is introduced for a visual-inertial navigation system to close the control loop directly at the Cartesian space by estimating the end-effector pose. Accordingly, the robot tip is equipped with an inertial measurement unit (IMU) and a stereo camera to provide task-space feedback information for the proposed observer. It is demonstrated through the Lyapunov stability theorem that the resulting closed-loop system under the proposed observer-based controller is globally asymptotically stable. Besides this notable merit (global asymptotic stability), the proposed control method eliminates the need to compute inverse kinematics and increases trajectory tracking accuracy in task-space. The effectiveness and accuracy of the proposed control scheme are evaluated through computer simulations, where the proposed control structure is applied to a 6 degrees-of-freedom long-reach hydraulic robot manipulator.

Figures (6)

• Figure 1: Sketch of the proposed visual-inertial-based controller.
• Figure 2: 3-D hydraulic manipulator model in Simscape MultibodyTM.
• Figure 3: 3D view of the desired, estimated, and actual trajectories.
• Figure 4: Time evolution of position and attitude estimation errors.
• Figure 5: Task-space position tracking error.