Dynamic object goal pushing with mobile manipulators through model-free constrained reinforcement learning

TL;DR

This work tackles dynamic object pushing with mobile manipulators under substantial uncertainty by learning a constrained reinforcement learning policy that controls both a mobile base and a mounted arm. Training uses a high-fidelity simulator with domain randomization and a constrained PPO framework to enforce safety and feasibility, while a balance-aware objective and surface-based reach targets enable stable, 3D pushing and object reorientation. The approach achieves high success in simulation (≈91%) and strong hardware performance (≥80%) across diverse object masses, shapes, and materials, while demonstrating robust contact switching and adaptation to object size. The results highlight practical robustness for real-world deployment and emphasize the importance of balancing constraints to prevent toppling and ensure safe manipulation.

Abstract

Non-prehensile pushing to move and reorient objects to a goal is a versatile loco-manipulation skill. In the real world, the object's physical properties and friction with the floor contain significant uncertainties, which makes the task challenging for a mobile manipulator. In this paper, we develop a learning-based controller for a mobile manipulator to move an unknown object to a desired position and yaw orientation through a sequence of pushing actions. The proposed controller for the robotic arm and the mobile base motion is trained using a constrained Reinforcement Learning (RL) formulation. We demonstrate its capability in experiments with a quadrupedal robot equipped with an arm. The learned policy achieves a success rate of 91.35% in simulation and at least 80% on hardware in challenging scenarios. Through our extensive hardware experiments, we show that the approach demonstrates high robustness against unknown objects of different masses, materials, sizes, and shapes. It reactively discovers the pushing location and direction, thus achieving contact-rich behavior while observing only the pose of the object. Additionally, we demonstrate the adaptive behavior of the learned policy towards preventing the object from toppling.

Figures (7)

• Figure 1: Dynamic object pushing with a quadrupedal manipulator. The proposed controller learns to push unknown objects towards different goals. The motions are included in the supplementary video https://youtu.be/wGAdPGVf9Ws?si=j9YNlEufzQIGlPz4.
• Figure 2: The control pipeline used for moving and reorienting an object to a planar goal (darker object). Push policy is the proposed controller.
• Figure 3: A) The object's position is set to the environment origin ($W$), the robot base position is randomly sampled within an origin-centered annulus (yellow-shaded area), and the object goal (dark rectangle) within a circular area (dashed line). The robot, object, and goal are spawned with a random yaw orientation. B) Sampling on the object surface encourages interaction with different parts of the object during training.
• Figure 4: Experimental validation of the proposed controller for sequentially moving and re-orienting an object between two goal poses. The robot pushes a plastic box of 6.4 kg from one goal to another. The goal poses are shown as green boxes. Snapshots of previous times are shown with lower opacity. The robot successfully goes around the object to push from the correct side towards the goal (1-2, 5).
• Figure 5: Distance and yaw angle error between object and goal. The shaded regions denote the time when the object is moved away from the goal.