Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multimodal Control of Manipulators: Coupling Kinematics and Vision for Self-Driving Laboratory Operations

Shifa Sulaiman, Amarnath H, Simon Bogh, Naresh Marturi

TL;DR

This work presents a modular framework for autonomous manipulation in self-driving laboratories by fusing real-time vision-based pose estimation with Jacobian-based motion planning. Trajectories are generated via RRT* and executed with a Damped Least Squares inverse kinematics scheme, underpinned by screw theory kinematics for a 7-DOF arm with a 3-finger gripper. The approach is evaluated through quantitative metrics including RMSE, velocity continuity, and higher-order motion profiles, demonstrating smooth, near-singularity-robust trajectories with sub-millimeter pose accuracy and sub-degree orientation error. The results validate the feasibility of vision-guided, clutter-free manipulation in SDL settings and establish a reproducible baseline for future extension to more complex objects, obstacles, and multi-arm coordination.

Abstract

Motion planning schemes are used for planning motions of a manipulator from an initial pose to a final pose during a task execution. A motion planning scheme generally comprises of a trajectory planning method and an inverse kinematic solver to determine trajectories and joints solutions respectively. In this paper, 3 motion planning schemes developed based on Jacobian methods are implemented to traverse a redundant manipulator with a coupled finger gripper through given trajectories. RRT* algorithm is used for planning trajectories and screw theory based forward kinematic equations are solved for determining joint solutions of the manipulator and gripper. Inverse solutions are computed separately using 3 Jacobian based methods such as Jacobian Transpose (JT), Pseudo Inverse (PI), and Damped Least Square (DLS) methods. Space Jacobian and manipulability measurements of the manipulator and gripper are obtained using screw theory formulations. Smoothness and RMSE error of generated trajectories and velocity continuity, acceleration profile, jerk, and snap values of joint motions are analysed for determining an efficient motion planning method for a given task. Advantages and disadvantages of the proposed motion planning schemes mentioned above are analysed using simulation studies to determine a suitable inverse solution technique for the tasks.

Multimodal Control of Manipulators: Coupling Kinematics and Vision for Self-Driving Laboratory Operations

TL;DR

This work presents a modular framework for autonomous manipulation in self-driving laboratories by fusing real-time vision-based pose estimation with Jacobian-based motion planning. Trajectories are generated via RRT* and executed with a Damped Least Squares inverse kinematics scheme, underpinned by screw theory kinematics for a 7-DOF arm with a 3-finger gripper. The approach is evaluated through quantitative metrics including RMSE, velocity continuity, and higher-order motion profiles, demonstrating smooth, near-singularity-robust trajectories with sub-millimeter pose accuracy and sub-degree orientation error. The results validate the feasibility of vision-guided, clutter-free manipulation in SDL settings and establish a reproducible baseline for future extension to more complex objects, obstacles, and multi-arm coordination.

Abstract

Motion planning schemes are used for planning motions of a manipulator from an initial pose to a final pose during a task execution. A motion planning scheme generally comprises of a trajectory planning method and an inverse kinematic solver to determine trajectories and joints solutions respectively. In this paper, 3 motion planning schemes developed based on Jacobian methods are implemented to traverse a redundant manipulator with a coupled finger gripper through given trajectories. RRT* algorithm is used for planning trajectories and screw theory based forward kinematic equations are solved for determining joint solutions of the manipulator and gripper. Inverse solutions are computed separately using 3 Jacobian based methods such as Jacobian Transpose (JT), Pseudo Inverse (PI), and Damped Least Square (DLS) methods. Space Jacobian and manipulability measurements of the manipulator and gripper are obtained using screw theory formulations. Smoothness and RMSE error of generated trajectories and velocity continuity, acceleration profile, jerk, and snap values of joint motions are analysed for determining an efficient motion planning method for a given task. Advantages and disadvantages of the proposed motion planning schemes mentioned above are analysed using simulation studies to determine a suitable inverse solution technique for the tasks.

Paper Structure

This paper contains 15 sections, 36 equations, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodologies
  4. Approach Overview
  5. Kinematic modeling of the manipulator
  6. Workspace analysis
  7. Determining Jacobian via screw theory
  8. Inverse Kinematics Techniques - Damped Least Squares (DLS) Strategy
  9. Vision-Based Grasping Framework
  10. Planar Object Localization and Pose Inference
  11. Motion evaluation metrics
  12. Results and Discussion
  13. Evaluation of Trajectory Motions
  14. Discussion
  15. Conclusion

Figures (9)

  • Figure S1: Methodology adopted in this work.
  • Figure S2: Illustration of various frames and screw axes of the used KUKA iiwa manipulator.
  • Figure S3: Views of combined workspace of the manipulator with hand (a)3D (b)Sectioned (c)Top.
  • Figure S4: Simulation environment (a) Gazebo (b) Rviz.
  • Figure S5: Vision algorithm outputs: (a) Bounding box, (b) Pose of object
  • ...and 4 more figures