Robustness-Aware Tool Selection and Manipulation Planning with Learned Energy-Informed Guidance

TL;DR

A robustness-aware method that jointly selects tools and plans contact-rich manipulation trajectories, explicitly optimizing for robustness against disturbances, using an energy-based robustness metric that guides the planner toward robust manipulation behaviors.

Abstract

Humans subconsciously choose robust ways of selecting and using tools, for example, choosing a ladle over a flat spatula to serve meatballs. However, robustness under external disturbances remains underexplored in robotic tool-use planning. This paper presents a robustness-aware method that jointly selects tools and plans contact-rich manipulation trajectories, explicitly optimizing for robustness against disturbances. At the core of our method is an energy-based robustness metric that guides the planner toward robust manipulation behaviors. We formulate a hierarchical optimization pipeline that first identifies a tool and configuration that optimizes robustness, and then plans a corresponding manipulation trajectory that maintains robustness throughout execution. We evaluate our method across three representative tool-use tasks. Simulation and real-world results demonstrate that our method consistently selects robust tools and generates disturbance-resilient manipulation plans.

Figures (8)

• Figure 1: How can the robot choose the best tool to scoop and lift a fish so that it does not fall out of the tool during manipulation? This work introduces a robustness-aware planner that selects and uses tools effectively under disturbances.
• Figure 2: Manipulation robustness in three scenarios. Fish in a shovel (a) and scissors on a treble hook (b) remain secured unless large disturbances occur; keyframes (1–3) illustrate minimal-energy escape paths used to quantify robustness. The robustness of a keyring in a snaplock (c) can also be characterized by clearance, i.e., the minimal dilation needed to completely prevent escape.
• Figure 3: Overview of the robustness-aware tool selection and manipulation planning method. The pipeline first performs keyframe optimization to jointly select the most robust tool and tool–object configuration. The best-found configuration then serves as a waypoint for full trajectory optimization, which plans a robustness-preserving manipulation strategy. A neural network trained offline on simulated robustness scores enables efficient robustness evaluation throughout the planning process.
• Figure 4: Our planner identifies the most robust tool from a cluttered table and plans an optimal trajectory to robustly pull a tape (a) or scoop a fish (b).
• Figure 5: Robustness of different tools under disturbances, evaluated in their best-found configurations ($s^*_{\text{obj}}$, $s^*_{\text{tool}}$) with MEE. For each tool, 100 simulated episodes of object trajectories under random force disturbances are shown in red. Tools are ordered from left to right by increasing manipulation robustness to external disturbances.