Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Gradient-Based Inference for Manipulation Planning in Contact Factor Graphs

Jeongmin Lee, Sunkyung Park, Minji Lee, Dongjun Lee

TL;DR

The paper addresses manipulation planning under complex contact and dynamics by introducing Contact Factor Graphs (CFG), a differentiable, factorized graphical model that encodes contact and quasi-dynamic constraints. It develops gradient-based inference methods, including convex MAP on conditional distributions via a semi-analytic primal solver and a score computation using the envelope theorem, plus a variational approach with Stein Variational Gradient Descent to capture multimodal posteriors. Key contributions include differentiable contact features, a convex optimization-based MAP framework with efficient Hessian factorization, and SVGD-based sampling to produce diverse solutions; demonstrated on stable placement, pivoting, valve turning, and multifinger grasp-and-place tasks. The framework enables fast, scalable sample generation and posterior approximation for planning under contact, with potential as a data-generation tool for learning-based models in robotics.

Abstract

This paper presents a framework designed to tackle a range of planning problems arise in manipulation, which typically involve complex geometric-physical reasoning related to contact and dynamic constraints. We introduce the Contact Factor Graph (CFG) to graphically model these diverse factors, enabling us to perform inference on the graphs to approximate the distribution and sample appropriate solutions. We propose a novel approach that can incorporate various phenomena of contact manipulation as differentiable factors, and develop an efficient inference algorithm for CFG that leverages this differentiability along with the conditional probabilities arising from the structured nature of contact. Our results demonstrate the capability of our framework in generating viable samples and approximating posterior distributions for various manipulation scenarios.

Efficient Gradient-Based Inference for Manipulation Planning in Contact Factor Graphs

TL;DR

The paper addresses manipulation planning under complex contact and dynamics by introducing Contact Factor Graphs (CFG), a differentiable, factorized graphical model that encodes contact and quasi-dynamic constraints. It develops gradient-based inference methods, including convex MAP on conditional distributions via a semi-analytic primal solver and a score computation using the envelope theorem, plus a variational approach with Stein Variational Gradient Descent to capture multimodal posteriors. Key contributions include differentiable contact features, a convex optimization-based MAP framework with efficient Hessian factorization, and SVGD-based sampling to produce diverse solutions; demonstrated on stable placement, pivoting, valve turning, and multifinger grasp-and-place tasks. The framework enables fast, scalable sample generation and posterior approximation for planning under contact, with potential as a data-generation tool for learning-based models in robotics.

Abstract

This paper presents a framework designed to tackle a range of planning problems arise in manipulation, which typically involve complex geometric-physical reasoning related to contact and dynamic constraints. We introduce the Contact Factor Graph (CFG) to graphically model these diverse factors, enabling us to perform inference on the graphs to approximate the distribution and sample appropriate solutions. We propose a novel approach that can incorporate various phenomena of contact manipulation as differentiable factors, and develop an efficient inference algorithm for CFG that leverages this differentiability along with the conditional probabilities arising from the structured nature of contact. Our results demonstrate the capability of our framework in generating viable samples and approximating posterior distributions for various manipulation scenarios.

Paper Structure

This paper contains 23 sections, 2 theorems, 16 equations, 7 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Planning as Inference
  4. Problems with Contact
  5. Modeling Contact Factor Graphs
  6. Problem Formulation
  7. Contact Mechanics
  8. Quasi-Dynamics
  9. Differentiable Contact Features
  10. Composition of Factors
  11. Inference in Contact Factor Graphs
  12. MAP Inference on the Conditional Distribution
  13. Conditional Independence and Factorization
  14. Inference on $q$
  15. Score Function via Envelope Theorem
  16. ...and 8 more sections

Key Result

Proposition 1

The condition eq:contact_coulomb is equivalent to the Karush-Kuhn-Tucker conditions derived from the principle of maximal dissipation macklin2019nonsmooth: while we slightly abuse notation here by denoting $\lambda_n$ as $n(q)^T\lambda$.

Figures (7)

  • Figure 1: Overview example of the proposed framework. Given the information on environment, contact factor graphs is composed using dynamics and contact factors relevant to the tasks. During the inference phase, convex optimization is employed to compress the distribution with respect to $q$. Score function is computed and applied within optimization or SVGD for inference.
  • Figure 2: Comparison our MAP inference algorithm and direct inference on the joint distribution. Left: Convergence over iterations. Right: Quality of the final results.
  • Figure 3: Examples of variational inference results in CFG. Left: Stable object poses sampled from the approximated distribution. Right: Visualization of the SVGD results under a fixed rotation.
  • Figure 4: Left: Motion of pivoting manipulation. Middle: Visualization of SVGD results within our framework. Right: Visualization of ensemble MCMC results.
  • Figure 5: Snapshots of planning results generated through inferences in CFG. Top: valve turning with slide maneuvers. Bottom: multifinger grasping and placing.
  • ...and 2 more figures

Theorems & Definitions (4)

  • Proposition 1
  • proof
  • Lemma 1
  • proof