Enhancing Adaptivity of Two-Fingered Object Reorientation Using Tactile-based Online Optimization of Deconstructed Actions

TL;DR

The paper tackles in-hand object reorientation with two-finger grippers under unknown object properties and nonlinear contact dynamics. It introduces a tactile-based three-action decomposition (Task-oriented, Constraint-based, Coordinating) and applies online gradient planning to adapt actions in real time, without requiring prior object models. Key contributions include reformulating planning around tactile expectations, decomposing complex actions into directly superimposable components, and demonstrating online optimization that handles unseen objects and environmental constraints. The results show robust performance across various everyday objects and scenarios, highlighting improved adaptability and dexterity in constrained environments.

Abstract

Object reorientation is a critical task for robotic grippers, especially when manipulating objects within constrained environments. The task poses significant challenges for motion planning due to the high-dimensional output actions with the complex input information, including unknown object properties and nonlinear contact forces. Traditional approaches simplify the problem by reducing degrees of freedom, limiting contact forms, or acquiring environment/object information in advance, which significantly compromises adaptability. To address these challenges, we deconstruct the complex output actions into three fundamental types based on tactile sensing: task-oriented actions, constraint-oriented actions, and coordinating actions. These actions are then optimized online using gradient optimization to enhance adaptability. Key contributions include simplifying contact state perception, decomposing complex gripper actions, and enabling online action optimization for handling unknown objects or environmental constraints. Experimental results demonstrate that the proposed method is effective across a range of everyday objects, regardless of environmental contact. Additionally, the method exhibits robust performance even in the presence of unknown contacts and nonlinear external disturbances.

Figures (6)

• Figure 1: The orange and green arrows represent the reorientation tasks for the contact and non-contact states, respectively. Both states follow a common method, initiated by the blue arrows. The tactile information is represent as $\boldsymbol{S}_1$ and $S_2$, which are used to control the actions. Of the three defined action types, task-oriented actions (when in contact with the environment) and coordinating actions (without environmental contact) do not require tactile feedback. These actions are optimized and learned online. Finally, different actions are superimposed to achieve the reorientation of the object.
• Figure 2: The reorientation task is divided into the object rotating target and the contact constraint condition. The former is achieved by task-based action, and the latter is achieved by constraint-based action.
• Figure 3: Reorientation with environment contact. The red arrows represent different actions.
• Figure 4: Objects (a)--(e) and their respective coordinate frames are shown. (f) is the initial and target states of object reorientation with environmental contact, while (g) shows the initial and target states of object reorientation without environmental contact.
• Figure 5: Results for the glue bottle. The black line represents the angle between the object's posture and the target posture. The task-oriented actions (red arrows) and coordinating actions (green arrows) represent the movements of the gripper, while the constraint-based action (blue line) shows the change in grasping force. The three types of actions are performed sequentially at different time points.