Learning to Double Guess: An Active Perception Approach for Estimating the Center of Mass of Arbitrary Objects

TL;DR

This work tackles the problem of estimating the center of mass for arbitrary objects in unstructured environments by introducing U-GRAPH, a framework that couples a Bayesian Neural Network for uncertainty quantification with an ActiveNet that selects informative second orientations through grid search. The two-measurement strategy uses an initial CoM estimate from a fixed grasp, followed by a second orientation to gain additional information, and fuses both measurements into a Gaussian posterior for a robust final estimate. Empirical evaluation on customized training objects and unseen real-world objects shows that U-GRAPH achieves an average error of about $1.47\,\text{cm}$ and $7.6\%$ relative error on unseen data, outperforming analytical baselines and simple two-measurement schemes. The method demonstrates strong generalization across objects with varying contact geometry, surface friction, and shapes, and outlines future improvements such as slip-detection with GelSight sensors to further enhance robustness.

Abstract

Manipulating arbitrary objects in unstructured environments is a significant challenge in robotics, primarily due to difficulties in determining an object's center of mass. This paper introduces U-GRAPH: Uncertainty-Guided Rotational Active Perception with Haptics, a novel framework to enhance the center of mass estimation using active perception. Traditional methods often rely on single interaction and are limited by the inherent inaccuracies of Force-Torque (F/T) sensors. Our approach circumvents these limitations by integrating a Bayesian Neural Network (BNN) to quantify uncertainty and guide the robotic system through multiple, information-rich interactions via grid search and a neural network that scores each action. We demonstrate the remarkable generalizability and transferability of our method with training on a small dataset with limited variation yet still perform well on unseen complex real-world objects.

Figures (5)

• Figure 1: We design an active perception algorithm to estimate the center of mass of arbitrary objects. Our algorithm uses the first estimation from the F/T reading to infer a new rotational orientation that improves the estimation, then executes the action and estimates again with a second F/T reading.
• Figure 2: Illustration of the simplified model of CoM on a real-world object. In our setup, we try to estimate the $\mathrm{d}x$, $\mathrm{d}y$, and $\mathrm{d}z$ which are the displacement of CoM away from the grasping point.
• Figure 3: a) Flowchart for training Bayesian Neural Network. We train BNN with Markov Chain Monte Carlo and No U-Turn Sampler iteratively. b) Flowchart for training an active perception module. We calculate the score from two orientations as the supervised label of the ActiveNet. We use the first prediction's mean and standard deviation along with the second angle as the input to the network. c) Flowchart for inferencing with U-GRAPH. The robot first grasps with a fixed orientation, then passes the F/T reading with (0, 0) as orientations into the BNN. We use ActiveNet and grid search to find the second action. We pass the second F/T reading with the orientation through BNN to get a secondary prediction and join that with the first prediction to form the posterior prediction.
• Figure 4: a) Example of a data collection robot grasping with the location of F/T Sensor and gripper. b) Printed data collection objects in the real world, and standard lab weights for training data collection. AprilTags are placed on all of the objects. We refer to the object on the left as Plate and the object on the right as Box.
• Figure 5: Mean error and mean standard deviation (shown with the error bar) of the estimated center of mass for customized objects obtained from different methods.