Learning Object Properties Using Robot Proprioception via Differentiable Robot-Object Interaction

TL;DR

This work addresses object parameter identification using robot proprioception, removing the need for external sensing or object tracking. It leverages differentiable simulations of robot-object interactions to infer object mass and elasticity by matching robot joint trajectories to forward models, using only joint encoder signals. The approach handles fixed joints, contact-based interactions, and deformable bodies, and demonstrates accurate estimates on a low-cost OpenManipulator-X, achieving rapid calibration on a laptop. The results point to a practical path for vision-free, data-efficient calibration of manipulated objects across diverse robotic platforms.

Abstract

Differentiable simulation has become a powerful tool for system identification. While prior work has focused on identifying robot properties using robot-specific data or object properties using object-specific data, our approach calibrates object properties by using information from the robot, without relying on data from the object itself. Specifically, we utilize robot joint encoder information, which is commonly available in standard robotic systems. Our key observation is that by analyzing the robot's reactions to manipulated objects, we can infer properties of those objects, such as inertia and softness. Leveraging this insight, we develop differentiable simulations of robot-object interactions to inversely identify the properties of the manipulated objects. Our approach relies solely on proprioception -- the robot's internal sensing capabilities -- and does not require external measurement tools or vision-based tracking systems. This general method is applicable to any articulated robot and requires only joint position information. We demonstrate the effectiveness of our method on a low-cost robotic platform, achieving accurate mass and elastic modulus estimations of manipulated objects with just a few seconds of computation on a laptop.

Figures (7)

• Figure 1: Calibrating Object Parameters through Differentiable Physics Using Proprioceptive Signals. Left: Our method aims to identify object parameters, such as the mass and material properties of the purple sphere. Middle: We utilize differentiable physics to simulate interactions between the robot and the object. Right: Object parameters are identified by supervising the differentiable physics simulation (top) using proprioceptive signals (joint positions, shown as green circles) from the real robot (bottom). Notably, our approach does not require tracking the object's trajectory (red circles); instead, it relies solely on the robot's internal sensors for the calibration process.
• Figure 2: Learning Object Properties through Various Robot-object Dynamics Enabled by Joint Activations. (a1)(a2) The object is attached to the robot as a fixed joint. The heavier ball on the left causes the robot to move less under the same joint torque. (b) The object interacts with the robot through contacts and collisions within a container. (c) The object deforms due to compression forces applied by the gripper.
• Figure 3: Proprioception—the robot’s internal sensing capabilities. Our approach constructs the optimization objective, i.e., \ref{['eqn:loss']}, using solely proprioceptive signals, which are directly available from the robot's internal encoders. This approach does not require exteroceptive signals, such as object motion tracking via external cameras.
• Figure 4: Closing the Sim-to-Real Gap through Accurate Object Property Identification. With the correctly identified object mass, our simulation closely matches real-world observations.
• Figure 5: Identifying Elastic Moduli. Our approach identifies the elastic moduli of materials with varying stiffness. With the learned parameters, the simulation closely matches the experiments.