A New Tightly-Coupled Dual-VIO for a Mobile Manipulator With Dynamic Locomotion

TL;DR

This work tackles instability in mobile manipulators during dynamic locomotion by introducing a tightly-coupled dual monocular VIO framework. Two independent VIO modules run in parallel at the base and end-effector, linked through an arm-odometry factor that leverages manipulator kinematics to provide soft geometric constraints, improving robustness and reducing drift. The method balances robustness and computational efficiency by avoiding a single oversized factor graph and instead applying moderate coupling between modules. Experimental results against a dual VINS-Mono baseline demonstrate significant improvements in translation accuracy and stability, especially under complex base-arm motions, with implications for active-SLAM and fault-tolerant, multi-VIO fusion in cluttered, dynamic environments.

Abstract

This paper introduces a new dual monocular visualinertial odometry (dual-VIO) strategy for a mobile manipulator operating under dynamic locomotion, i.e. coordinated movement involving both the base platform and the manipulator arm. Our approach has been motivated by challenges arising from inaccurate estimation due to coupled excitation when the mobile manipulator is engaged in dynamic locomotion in cluttered environments. The technique maintains two independent monocular VIO modules, with one at the mobile base and the other at the end-effector (EE), which are tightly coupled at the low level of the factor graph. The proposed method treats each monocular VIO with respect to each other as a positional anchor through arm-kinematics. These anchor points provide a soft geometric constraint during the VIO pose optimization. This allows us to stabilize both estimators in case of instability of one estimator in highly dynamic locomotions. The performance of our approach has been demonstrated through extensive experimental testing with a mobile manipulator tested in comparison to running dual VINS-Mono in parallel. We envision that our method can also provide a foundation towards active-SLAM (ASLAM) with a new perspective on multi-VIO fusion and system redundancy.

Figures (7)

• Figure 1: Our mobile manipulator comprises a 3- SUMMIT mobile base and a 7- WAM robotic arm . It has two cameras configured with only monocular- and ; one located at the and the other at the base . A system is used for GT.
• Figure 2: Frames and their relations: Solid arrows show constrained transformation; dashed arrows show unconstrained transformation; dotted lines show time-varying trajectories with a starting knot. To simplify matters, we assume that the state estimation is performed at the camera's IMU frame for both the base and the (i.e. $\chi^w_{b_B}, \chi^w_{b_E}\in{\textbf{SE}({3})}$). The kinematic parameters for the arm are calibrated offline based on the and 3D model at its zero-configuration, where the arm is straightened upwards ($g_{st}(\mathbf{0})\in{\textbf{SE}({3})}$).
• Figure 3: Illustration of the sliding window dual factor graph structure with the arm odometry factor coupling two states, with $\mathbf{x}_{b_k}$ as the primary state and $\mathbf{x}'_{b_k}$ as the corresponding secondary state at frame $k$. The bottom timeline shows an approximated representation of measurement timings and sampling strategy; data is sampled and pre-integrated between two frames, images are software synchronized, and arm joint states are sampled at each frame.
• Figure 4: The environment of a cluttered facility for experiments; uneven terrain, reflective glass window, low-feature illuminated ceiling, and repetitive robots. The green masking tape () on the floor helps operators locate the robot's starting position without collision during the experiment.
• Figure 5: A collection of six different arm motions with base fixed. (a) showcases the home position of the arm without motion; (b), (c), and (d) demonstrate static arm motion by moving the arm to the desired configuration (i.e. down, forward, up) initially and retracting back to the home (a) at the end of the base movements. (e) and (f) describe dynamic oscillating motions, by sweeping the arm left-and-right and up-and-down, respectively.