Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization

TL;DR

This work tackles robust pivoting manipulation under uncertainty by introducing a frictional stability margin and a Contact Implicit Bilevel Optimization (CIBO) framework that maximizes the worst-case margin along a manipulation trajectory. By deriving margin bounds for uncertainties in mass, CoM location, friction, finger contact, and patch contact, the authors formulate a bilevel optimization where lower-level LPs compute per-step margins and the upper level seeks to maximize the trajectory-wide robustness. The approach is extended to non-convex objects via mode-based optimization and patch contact, with validation through simulations and hardware on a 6 DoF manipulator, including MPC-based closed-loop control. The results show that CIBO yields more robust pivoting trajectories than baseline trajectory optimization, enabling reliable manipulation of diverse objects and resilience to disturbances, frictional variations, and sensing uncertainties. This framework paves the way for generalizable, friction-aware manipulation in uncertain real-world environments.

Abstract

Generalizable manipulation requires that robots be able to interact with novel objects and environment. This requirement makes manipulation extremely challenging as a robot has to reason about complex frictional interactions with uncertainty in physical properties of the object and the environment. In this paper, we study robust optimization for planning of pivoting manipulation in the presence of uncertainties. We present insights about how friction can be exploited to compensate for inaccuracies in the estimates of the physical properties during manipulation. Under certain assumptions, we derive analytical expressions for stability margin provided by friction during pivoting manipulation. This margin is then used in a Contact Implicit Bilevel Optimization (CIBO) framework to optimize a trajectory that maximizes this stability margin to provide robustness against uncertainty in several physical parameters of the object. We present analysis of the stability margin with respect to several parameters involved in the underlying bilevel optimization problem. We demonstrate our proposed method using a 6 DoF manipulator for manipulating several different objects. We also design and validate an MPC controller using the proposed algorithm which can track and regulate the position of the object during manipulation.

Figures (24)

• Figure 1: We consider the problem of reorienting parts for assembly using pivoting manipulation primitive. Such reorientation could possibly be required when the parts being assembled are too big to grasp in the initial pose (such as the gears) or the parts to be inserted during assembly are not in the desired pose (such as the pegs). The figure shows some instances during the implementation of our controller to reorient a gear and a peg.
• Figure 2: A schematic showing the free-body diagram of a rigid body during pivoting manipulation when the relative angle between $F_W$ and $F_S$ is zero. Point $P$ is the contact point with a manipulator. The black circle represents the origin of each frame. The object experiences four forces corresponding to two friction forces from external contact points $A$ and $B$, one control input $f_P$ from the manipulator at point $P$, and gravity at point $C$.
• Figure 3: A schematic showing the frame definition of a rigid body during pivoting manipulation. $F_W$, $F_S$, $F_O$, and $F_B$ are the world frame, slope frame, object frame, and frame at contact location $B$, respectively. Gravity is defined in $F_W$ where the gravity is parallel to $y$-axis of $F_W$. Pivoting manipulation happens with extrinsic contact $A$ and $B$ defined in $F_S$. $F_O$ is fixed with CoM of an object. $F_B$ is in parallel to $F_S$ with offset $B^S_x$ along $x$-axis of $F_S$. We also show an example of $i_x^\Sigma$ and $i_x^\Sigma$ in Table \ref{['tab:notion']}. In this example, $C_x^B$ and $C_y^B$ are illustrated.
• Figure 4: A schematic showing the free-body diagram of a rigid body during pivoting manipulation. We consider the stability margin of finger location due to imperfect control of stiffness controller in a robotic manipulator.
• Figure 5: A schematic showing the free-body diagram of a rigid body during pivoting manipulation with patch contact. We approximate patch contact as two point contacts $P_1$ and $P_2$ with the same force distribution. We assume that $P_1$ always lies on the vertex of the object for this simplistic patch contact model. $s$ is the distance between point contact $P_1$ and $P_2$ along $y$-axis of $F_O$.