Model-Based Adaptive Precision Control for Tabletop Planar Pushing Under Uncertain Dynamics

TL;DR

This work tackles non-prehensile tabletop pushing under uncertain dynamics by learning a single GRU-based push dynamics model and coupling it with a sampling-based MPPI controller. The method enables side-switching and variable-length pushes, supporting tasks such as precise posing, trajectory following, and obstacle avoidance without retraining. Domain randomization and sim-to-real transfer are demonstrated on a Franka Panda robot, with ablations and real-world experiments showing high precision and robust performance. The approach provides a flexible, task-agnostic framework for planar pushing, with future work focusing on scaling push ranges, improving sampling efficiency, and integrating perception for broader generalization.

Abstract

Data-driven planar pushing methods have recently gained attention as they reduce manual engineering effort and improve generalization compared to analytical approaches. However, most prior work targets narrow capabilities (e.g., side switching, precision, or single-task training), limiting broader applicability. We present a model-based framework for non-prehensile tabletop pushing that uses a single learned model to address multiple tasks without retraining. Our approach employs a recurrent GRU-based architecture with additional non-linear layers to capture object-environment dynamics while ensuring stability. A tailored state-action representation enables the model to generalize across uncertain dynamics, variable push lengths, and diverse tasks. For control, we integrate the learned dynamics with a sampling-based Model Predictive Path Integral (MPPI) controller, which generates adaptive, task-oriented actions. This framework supports side switching, variable-length pushes, and objectives such as precise positioning, trajectory following, and obstacle avoidance. Training is performed in simulation with domain randomization to support sim-to-real transfer. We first evaluate the architecture through ablation studies, showing improved prediction accuracy and stable rollouts. We then validate the full system in simulation and real-world experiments using a Franka Panda robot with markerless tracking. Results demonstrate high success rates in precise positioning under strict thresholds and strong performance in trajectory tracking and obstacle avoidance. Moreover, multiple tasks are solved simply by changing the controller's objective function, without retraining. While our current focus is on a single object type, we extend the framework by training on wider push lengths and designing a balanced controller that reduces the number of steps for longer-horizon goals.

Figures (15)

• Figure 1: The real-robot setup. The robot holds a stick with a small ball at the end to interact with the objects. A RGBD camera is used for perception.
• Figure 2: Push dynamics model architecture. The ($RO_t$, $\Delta RO_t$) parts of the input constitute the action.
• Figure 3: State-action representation. $O_t$ represents the 2D object pose before the push and $O_{t+1}$ represents it after the push. $RO_t$ represents the position of the pusher with respect to object before the push and $\Delta RO_t$ represents the pusher motion. $\Delta X_{t+1}$ and $\Delta \theta_{t+1}$ denote the changes in the object's position and orientation, respectively, after the push, and $\Delta O_{t+1} = (\Delta X_{t+1}, \Delta \theta_{t+1})$.
• Figure 4: Data collection process.This figure illustrates the data collection process over 10,000 episodes. Actions are sampled with directions uniformly distributed between $-45^\circ$ and $45^\circ$, and push magnitudes ranging from 2 mm to 3 cm. The red dots indicate the boundary of relative poses where actions are applied, designed to prevent premature contact (i.e., touching) before the action is executed.
• Figure 5: Simulation environment. The figure shows an episode of the precision positioning task, where the object must be pushed to its goal pose using the pusher. In the simulation, the yellowish cube represents the current object pose, the greenish marker indicates the target pose, and the blue sphere represents the pusher.