An Optimal Control Formulation of Tool Affordance Applied to Impact Tasks

TL;DR

This work tackles impact-aware tool use by formulating a constrained optimal control problem that incorporates tool affordances and directional dexterity. It develops an ADMM-iLQR solver to handle inequality constraints while planning grasp and manipulation as a viapoint problem, and introduces a directional velocity manipulability cost to bias postures toward task-efficient momentum transfer. The approach is validated through simulations in 2D/3D scenarios and on a real 7-DoF Franka Emika robot performing hammering with a pilot hole, where directional manipulability yielded superior hammering performance compared with baselines including common manipulability, tracking a desired ellipsoid, and human demonstrations. The results demonstrate the practical impact of leveraging tool affordances and directionally informed manipulation in complex impact tasks, with potential applicability to a broad range of tool-based robotic manipulation problems.

Abstract

Humans use tools to complete impact-aware tasks such as hammering a nail or playing tennis. The postures adopted to use these tools can significantly influence the performance of these tasks, where the force or velocity of the hand holding a tool plays a crucial role. The underlying motion planning challenge consists of grabbing the tool in preparation for the use of this tool with an optimal body posture. Directional manipulability describes the dexterity of force and velocity in a joint configuration along a specific direction. In order to take directional manipulability and tool affordances into account, we apply an optimal control method combining iterative linear quadratic regulator(iLQR) with the alternating direction method of multipliers(ADMM). Our approach considers the notion of tool affordances to solve motion planning problems, by introducing a cost based on directional velocity manipulability. The proposed approach is applied to impact tasks in simulation and on a real 7-axis robot, specifically in a nail-hammering task with the assistance of a pilot hole. Our comparison study demonstrates the importance of maximizing directional manipulability in impact-aware tasks.

Figures (16)

• Figure 1: Left: The ways in which tools are seized and used vary according to the characteristics of the task (tool affordances). For hammering a nail, we grasp the hammer in a way that is efficient for the task demands, since the greater the velocity, the deeper the nail can be driven. For nail-pulling, the same hammer is grasped in a different way. Here, the greater the static force applied to the nail, the easier the nail can be pulled out. Right: In a peg-in-hole task, we typically choose a comfortable pose to insert a peg into a narrow hole, which depends on the relations between the hand, the peg and the hole (with maximum force manipulability along the insertion direction).
• Figure 2: Illustration of a nail driven into a plastic foam. (a) shows the situation without a preformed insertion shape (pilot hole), where sideslip problems can occur; (b) shows the typical situation with a pilot hole, where the nail can be driven more easily so that it remains straight.
• Figure 3: Influence of the grasping pose on the velocity manipulability of a 3-axis planar arm, where the last link with light color represents the tool. In subfigure (a), a planar arm grasps a tool with the same posture but a different grasping position (left and right plot) and grasping orientation (left and middle plot). The number shows the projection value along the world frame and local end-effector frame (dashed box). Subfigure (b) shows the robot grasping the same tool with different joint angle configurations.
• Figure 4: ADMM-iLQR applied to a viapoint task with a desired range task. In subplot (a), we show four examples of the viapoint with a desired range, where the grey, red and blue robots represent the initial, via and final states of the robot, respectively. The first plot requires the robot to reach the range in cyan and then the final position (red ring). We increase the complexity of the task by setting the final pose specified after passing through the range, arriving at the range while maintaining a desired pose, and finally combining both constraints above into the fourth task scenario. The arrows in the last three plots (blue: viapoint with a desired range; red: final) represent the specified direction. In subplot (b), our approach is verified on a simulated Franka Emika robot.
• Figure 5: ADMM-iLQR applied to a pick-and-place task. In (a), we show the results of considering the maximization of manipulability, where the grey, red and blue robots correspond to the descriptions in \ref{['Fig3_1']}. The robot is required to pick the blue cube and place it in the black dashed cube region while maximizing the velocity manipulability of the robot-tool system along the object orientation. The arrow in the last plot represents the desired direction and the red dot represents the object tip. In (b), we display the velocity manipulability of the robot-tool system separately on the tool tip generated by different strategies. The projections along the desired direction generated by the different approaches are represented with arrows of different line styles. To observe the differences between each projection more clearly, we lined them up in parallel. For a better view of the overlayed ellipses, see the color version of this article.