Diffusion-Informed Probabilistic Contact Search for Multi-Finger Manipulation

TL;DR

This work presents Diffusion-Informed Probabilistic Contact Search (DIPS), which uses an A* search to plan a sequence of contact modes informed by a diffusion model and uses a particle filter-inspired method to reason about variability in diffusion sampling arising from model error.

Abstract

Planning contact-rich interactions for multi-finger manipulation is challenging due to the high-dimensionality and hybrid nature of dynamics. Recent advances in data-driven methods have shown promise, but are sensitive to the quality of training data. Combining learning with classical methods like trajectory optimization and search adds additional structure to the problem and domain knowledge in the form of constraints, which can lead to outperforming the data on which models are trained. We present Diffusion-Informed Probabilistic Contact Search (DIPS), which uses an A* search to plan a sequence of contact modes informed by a diffusion model. We train the diffusion model on a dataset of demonstrations consisting of contact modes and trajectories generated by a trajectory optimizer given those modes. In addition, we use a particle filter-inspired method to reason about variability in diffusion sampling arising from model error, estimating likelihoods of trajectories using a learned discriminator. We show that our method outperforms ablations that do not reason about variability and can plan contact sequences that outperform those found in training data across multiple tasks. We evaluate on simulated tabletop card sliding and screwdriver turning tasks, as well as the screwdriver task in hardware to show that our combined learning and planning approach transfers to the real world.

Figures (6)

• Figure 1: DIPS plans contact interactions that turn the screwdriver from a) to b), where it is regrasped to allow for further turning in c). Contact points are shown in red, with empty circles for target contacts. The yellow arrows show screwdriver turning and green show finger motion.
• Figure 2: Offline, we sample contact sequences from a designed prior. We generate a dataset $\mathcal{D}$ of trajectories in simulation. We train a diffusion model $M$ and discriminator $\Psi$ on $\mathcal{D}$. Online, we plan a contact sequence $C$ given a state $\mathbf{s}_t$. We expand nodes in blue corresponding to contact mode sequences and inform the search using a distribution $p_C(\bm{\tau})$ parameterized with a set of trajectories $\mathcal{P}$ in pink. We diffuse trajectories conditioned on the child node's contact mode and evaluate them with $\Psi$. The dotted lines are samples discarded in the resampling used to update $p_C(\bm{\tau})$. Given a single contact mode and $\mathbf{s}_t$, we optimize a trajectory of length $H$ initialized with samples from $M$. We rerun the trajectory optimization every timestep. After each contact mode, we replan $C$.
• Figure 3: a) Simulated card and b) Simulated screwdriver environments. The blue valve in b) is for visualization only and has no collision geometry.
• Figure 4: Simulated Card results over 10 trials.
• Figure 5: Simulated Screwdriver results over 10 trials.