Object-Reconstruction-Aware Whole-body Control of Mobile Manipulators

Abstract

Object reconstruction and inspection tasks play a crucial role in various robotics applications. Identifying paths that reveal the most unknown areas of the object is paramount in this context, as it directly affects reconstruction efficiency. Current methods often use sampling based path planning techniques, evaluating views along the path to enhance reconstruction performance. However, these methods are computationally expensive as they require evaluating several candidate views on the path. To this end, we propose a computationally efficient solution that relies on calculating a focus point in the most informative region and having the robot maintain this point in the camera field of view along the path. In this way, object reconstruction related information is incorporated into the whole body control of a mobile manipulator employing a visibility constraint without the need for an additional path planner. We conducted comprehensive and realistic simulations using a large dataset of 114 diverse objects of varying sizes from 57 categories to compare our method with a sampling based planning strategy and a strategy that does not employ informative paths using Bayesian data analysis. Furthermore, to demonstrate the applicability and generality of the proposed approach, we conducted real world experiments with an 8 DoF omnidirectional mobile manipulator and a legged manipulator. Our results suggest that, compared to a sampling based strategy, there is no statistically significant difference in object reconstruction entropy, and there is a 52.3% probability that they are practically equivalent in terms of coverage. In contrast, our method is 6.2 to 19.36 times faster in terms of computation time and reduces the total time the robot spends between views by 13.76% to 27.9%, depending on the camera FoV and model resolution.

Figures (17)

• Figure 1: First, an NBV is calculated. Then, the robot moves to the NBV while focusing on a Focus Point and avoiding obstacles in the workspace, enabling the robot to reveal more unknown areas safely.
• Figure 2: Illustration of the general framework, in which the dashed box highlights our contribution. Given the current partial object model and a set of candidate views, an NBV algorithm (RSV) Ref:Isler selects the next view to visit. Our method then uses this target NBV and the current partial model to select an informative focus point and employ a whole-body control strategy to reach the target while keeping the focus point within the camera FoV with a visibility constraint Ref:Dursun2023, avoiding obstacles with vector field inequalities (VFIs) Ref:VFI_Marinho, and mitigating local minima at obstacle boundaries with a circulation constraint Ref:Circulation.
• Figure 3: A 2D representation of the general concept adopted by sampling-based IPP strategies.
• Figure 4: The search space for candidate view generation is defined by 40 positions evenly distributed around a cylinder with a radius of 3 m. Each position has five orientations, resulting in a search space comprising 200 views.
• Figure 5: Illustration of the stages for focus point calculation. Steps 1–5 are repeated until the robot reaches the NBV.