Correlation-Aware Dual-View Pose and Velocity Estimation for Dynamic Robotic Manipulation

TL;DR

This work tackles robust pose and velocity estimation for dynamic robotic manipulation using dual-vision sensors. It introduces a correlation-aware, decentralized fusion framework built on matrix Lie groups, with two adaptive EKFs operating on $SE(3)\times \mathbb{R}^3 \times \mathbb{R}^3$ and updating on $SE(3)$, fused via a closed-form Lie-group fusion rule. Key contributions include the adaptive Lie-group EKF, an augmented $SE(3)\times \mathbb{R}^3 \times \mathbb{R}^3$ target-dynamics model, and a correlation-aware fusion strategy for dual-view measurements from eye-in-hand and eye-to-hand configurations. Experimental validation on a UFactory $x$Arm 850 with RealSense cameras demonstrates improved accuracy and robustness over state-of-the-art dual-view approaches, particularly under high acceleration and intermittent sensing conditions. The results suggest practical impact for reliable real-time manipulation and planning in cluttered or dynamic environments, with future work aimed at non-Gaussian noise handling and marker-free object pose estimation.

Abstract

Accurate pose and velocity estimation is essential for effective spatial task planning in robotic manipulators. While centralized sensor fusion has traditionally been used to improve pose estimation accuracy, this paper presents a novel decentralized fusion approach to estimate both pose and velocity. We use dual-view measurements from an eye-in-hand and an eye-to-hand vision sensor configuration mounted on a manipulator to track a target object whose motion is modeled as random walk (stochastic acceleration model). The robot runs two independent adaptive extended Kalman filters formulated on a matrix Lie group, developed as part of this work. These filters predict poses and velocities on the manifold $\mathbb{SE}(3) \times \mathbb{R}^3 \times \mathbb{R}^3$ and update the state on the manifold $\mathbb{SE}(3)$. The final fused state comprising the fused pose and velocities of the target is obtained using a correlation-aware fusion rule on Lie groups. The proposed method is evaluated on a UFactory xArm 850 equipped with Intel RealSense cameras, tracking a moving target. Experimental results validate the effectiveness and robustness of the proposed decentralized dual-view estimation framework, showing consistent improvements over state-of-the-art methods.

Figures (7)

• Figure 1: Coordinate frames of the manipulator, cameras, and the target objects.
• Figure 2: Estimation and fusion framework of the manipulator.
• Figure 3: The setup: (a) front view of the cameras, (b) the rover carrying the object, (c) the arm following the target, and (d) camera calibration.
• Figure 4: Results of scenario I; (a) target trajectory, and (b) its velocity.
• Figure 5: Results of scenario II; (a) target trajectory, and (b) its velocity.