Push Anything: Single- and Multi-Object Pushing From First Sight with Contact-Implicit MPC

TL;DR

Push Anything presents a real-time planar pushing pipeline that handles diverse objects and clutter by fusing real-world perception, robust tracking, and a fast CI-MPC controller. The core methodological advance is C3+, an ADMM-based enhancement of Consensus Complementarity Control that replaces costly complementarity projections with fast, decoupled 1D optimizations, enabling multi-object reasoning and long-horizon planning. Hardware experiments demonstrate high success rates across 33 objects and multi-object scenarios, with significantly faster solve times than prior CI-MPC methods. The work advances practical non-prehensile manipulation by closing the perception-to-action loop and validating a scalable approach to cluttered, contact-rich manipulation in real hardware.

Abstract

Non-prehensile manipulation of diverse objects remains a core challenge in robotics, driven by unknown physical properties and the complexity of contact-rich interactions. Recent advances in contact-implicit model predictive control (CI-MPC), with contact reasoning embedded directly in the trajectory optimization, have shown promise in tackling the task efficiently and robustly, yet demonstrations have been limited to narrowly curated examples. In this work, we showcase the broader capabilities of CI-MPC through precise planar pushing tasks over a wide range of object geometries, including multi-object domains. These scenarios demand reasoning over numerous inter-object and object-environment contacts to strategically manipulate and de-clutter the environment, challenges that were intractable for prior CI-MPC methods. To achieve this, we introduce Consensus Complementarity Control Plus (C3+), an enhanced CI-MPC algorithm integrated into a complete pipeline spanning object scanning, mesh reconstruction, and hardware execution. Compared to its predecessor C3, C3+ achieves substantially faster solve times, enabling real-time performance even in multi-object pushing tasks. On hardware, our system achieves overall 98% success rate across 33 objects, reaching pose goals within tight tolerances. The average time-to-goal is approximately 0.5, 1.6, 3.2, and 5.3 minutes for 1-, 2-, 3-, and 4-object tasks, respectively. Project page: https://dairlab.github.io/push-anything.

Figures (8)

• Figure 1: Experimental Setup: The Franka Emika Panda arm uses a spherical end effector to push and rearrange four objects from an initial cluttered configuration.
• Figure 2: System diagram of the Push Anything framework.
• Figure 3: Illustration of sampling-based CI-MPC on the planar Push-T task chi2023diffusionpolicy. Different end effector positions are shown with their associated MPC costs. The black solid lines denote the current pose of the T, and the red dashed lines denote the target pose. The optimal trajectory requires translating the T upward and to the right while rotating it clockwise. However, executing short-horizon MPC from the current end effector position (red circle) produces the opposite effect, pushing the T downward and counter-clockwise and resulting in higher cost. In contrast, the green candidate position yields the lowest cost, as it enables MPC to effectively align the T with its target. Thus, moving the end effector first to this green location before executing MPC facilitates more effective manipulation.
• Figure 4: Visualization of the sampling strategy for end effector locations. The gray plane indicates the ground, and the orange planes represent local tangent planes to the mesh surfaces. Blue arrows project surface samples outwards from the mesh along the face normals, then purple arrows project those to a fixed height in the world, generating candidate samples (green dots). Samples located too close to object surfaces (e.g. red dot) are discarded.
• Figure 5: Projection step in C3+. Dashed lines indicate paths from the original $(\lambda_{\circ}, \eta_{\circ})$ (blue circles) to the projected $(\delta_k^\lambda, \delta_k^\eta)^\ast$ (yellow circles) onto the feasible complementarity set (orange lines).