Tracing Energy Flow: Learning Tactile-based Grasping Force Control to Prevent Slippage in Dynamic Object Interaction

TL;DR

The paper tackles the challenge of minimizing slip in dynamic, multi-contact grasping when object properties and external sensing are unknown. It introduces a physics-informed energy abstraction that models the object as a virtual energy container and derives energy-consistency signals from tactile input to infer slip, integrated into a Fourier-featured linear Gaussian model (LGM-FF) for scalable dynamics and a probabilistic Model Predictive Controller (pMPC) for real-time grasp force planning. The approach enables learning from scratch within minutes and achieves robust slip reduction across diverse object-motion pairs, validated in both simulation and hardware without relying on object priors or external sensing. This tactile-driven framework offers a sample-efficient pathway to reliable dexterous manipulation in uncertain environments, with practical implications for robotic manipulation in unstructured settings.

Abstract

Regulating grasping force to reduce slippage during dynamic object interaction remains a fundamental challenge in robotic manipulation, especially when objects are manipulated by multiple rolling contacts, have unknown properties (such as mass or surface conditions), and when external sensing is unreliable. In contrast, humans can quickly regulate grasping force by touch, even without visual cues. Inspired by this ability, we aim to enable robotic hands to rapidly explore objects and learn tactile-driven grasping force control under motion and limited sensing. We propose a physics-informed energy abstraction that models the object as a virtual energy container. The inconsistency between the fingers' applied power and the object's retained energy provides a physically grounded signal for inferring slip-aware stability. Building on this abstraction, we employ model-based learning and planning to efficiently model energy dynamics from tactile sensing and perform real-time grasping force optimization. Experiments in both simulation and hardware demonstrate that our method can learn grasping force control from scratch within minutes, effectively reduce slippage, and extend grasp duration across diverse motion-object pairs, all without relying on external sensing or prior object knowledge.

Figures (11)

• Figure 1: Overview of the proposed framework for real-time grasping force control under dynamic object interaction using tactile sensing. The object is abstracted as an energy container with unknown properties, where fingertip-applied power and retained energy change are compared to infer energy loss as a physically grounded indicator of slippage. These energy quantities are formulated into an energy-state representation, which is used in a Model-based Reinforcement Learning (MBRL) framework to learn energy-flow dynamics and perform grasping force optimization under dynamic object interaction via probabilistic Model Predictive Control (pMPC) to reduce slippage. The system is validated in both simulation and hardware.
• Figure 2: Energy abstraction from multiple contact interaction. We compute the total applied power from all fingertip contacts and compare it with the object’s retained power. Inconsistent mass estimates reflect energy loss, which can be attributed to slippage. This physics-informed abstraction enables slip-aware stability assessment without requiring object observation.
• Figure 3: Robotic hands and manipulated objects in the hardware and simulated environments.
• Figure 4: Overview of our control framework integrating the energy abstraction and MBRL. The system consists of a nominal motion generator, a grasping force planner, an impedance controller, and a model trainer. The motion generator produces baseline Grasp Centroid (GC) and finger trajectories to induce object motion. The energy abstraction constructs a compact energy-state representation from tactile sensing, which is used by the MBRL to learn interaction dynamics. Real-time grasping force optimization is performed by pMPC based on the learned energy dynamics. This structure supports real-time grasping force control with tactile sensing only.
• Figure 5: Definition of periodic manipulation motions. The reference positions of the grasp centroid are denoted by $\boldsymbol{C}^\star$ (translation) and $\boldsymbol{\Theta}^\star$ (rotation). Each motion is described by a sinusoidal profile with angular frequency $\omega$ and amplitudes $l_{lift}$, $l_{circ}$, and $\theta_{rot}$ for Z-lift, XZ-circle, and Y-rotate respectively. The elevation offset $l_{\text{init}}$ denotes the lifted position during transition, and $t’ = t_a - 4$ indicates the relative time from the lift phase to the onset of manipulation. Parameters differ slightly between simulation and hardware to accommodate the kinematic constraints of each robot hand.