Dynamic Manipulation of Deformable Objects using Imitation Learning with Adaptation to Hardware Constraints

TL;DR

This work tackles dynamic manipulation of deformable objects by learning from a single human demonstration and bridging the gap between human motion and robot hardware limits. It introduces a two-stage imitation learning framework: a constraint-aware dynamic primitive encoded with Constrained Dynamic Movement Primitives (CDMPs) to rapidly alter the object state, followed by quasi-static refinement motions to fine-tune the result. The approach is instantiated in BILBO (Bimanual dynamic manipulation using Imitation Learning for Bag Opening), which uses Opt-DMP to map demonstrations to feasible robot actions and a linear refinement to optimize rim roundness, evaluated with novel bag-volume and rim-area metrics derived from alpha-shapes and convex hulls. Results show that a single demonstration can generalize to opening bags of varying sizes and materials, often achieving targets in a handful of actions, with refinement improving the final rim elongation and opening quality.

Abstract

Imitation Learning (IL) is a promising paradigm for learning dynamic manipulation of deformable objects since it does not depend on difficult-to-create accurate simulations of such objects. However, the translation of motions demonstrated by a human to a robot is a challenge for IL, due to differences in the embodiments and the robot's physical limits. These limits are especially relevant in dynamic manipulation where high velocities and accelerations are typical. To address this problem, we propose a framework that first maps a dynamic demonstration into a motion that respects the robot's constraints using a constrained Dynamic Movement Primitive. Second, the resulting object state is further optimized by quasi-static refinement motions to optimize task performance metrics. This allows both efficiently altering the object state by dynamic motions and stable small-scale refinements. We evaluate the framework in the challenging task of bag opening, designing the system BILBO: Bimanual dynamic manipulation using Imitation Learning for Bag Opening. Our results show that BILBO can successfully open a wide range of crumpled bags, using a demonstration with a single bag. See supplementary material at https://sites.google.com/view/bilbo-bag.

Figures (8)

• Figure 1: Our framework combines a dynamic motion primitive that adheres to the robots' constraints, for quick task progression, with quasi-static motions for final adjustments. We evaluate the framework by implementing BILBO: Bimanual dynamic manipulation using Imitation Learning for Bag Opening, a system for a practical bag-opening task.
• Figure 2: Estimation of the opening area using the $\alpha$-shape (orange), and convex hull (black). In a) the convex hull overestimates the rim area. In b) the $\alpha$-shape underestimates the area due to the occlusion of markers.
• Figure 3: Examples of two initial bag configurations used in the experiments and an example the final state after BILBO.
• Figure 4: Plastic bags used in the experiments.
• Figure 5: Quantitative results of the learned dynamic primitives using Opt-DMP, tau-DMP, and TC-DMP for three different bags in a) easy and b) hard initial configuration. The results show the area, volume, elongation and their target limits.