MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation

TL;DR

MoMaGen reframes automated demonstration generation for multi-step bimanual mobile manipulation as constrained optimization, enforcing hard constraints on reachability and object visibility while allowing soft constraints to shape navigation and retraction. By incorporating full-body motion (base, torso, arms, and head camera) and domain randomization, it generates highly diverse synthetic demonstrations from a single seed example. The approach yields richer data than prior methods, enabling imitation-learning policies that train effectively from synthetic data and require minimal real-world demonstrations for deployment, with evidence of cross-embodiment transfer and sim-to-real benefits. This framework offers a principled, scalable pathway to robust visuomotor policy learning for mobile manipulators in complex household environments, while highlighting practical considerations like scene-knowledge requirements and computational costs.

Abstract

Imitation learning from large-scale, diverse human demonstrations has proven effective for training robots, but collecting such data is costly and time-consuming. This challenge is amplified for multi-step bimanual mobile manipulation, where humans must teleoperate both a mobile base and two high-degree-of-freedom arms. Prior automated data generation frameworks have addressed static bimanual manipulation by augmenting a few human demonstrations in simulation, but they fall short for mobile settings due to two key challenges: (1) determining base placement to ensure reachability, and (2) positioning the camera to provide sufficient visibility for visuomotor policies. To address these issues, we introduce MoMaGen, which formulates data generation as a constrained optimization problem that enforces hard constraints (e.g., reachability) while balancing soft constraints (e.g., visibility during navigation). This formulation generalizes prior approaches and provides a principled foundation for future methods. We evaluate MoMaGen on four multi-step bimanual mobile manipulation tasks and show that it generates significantly more diverse datasets than existing methods. Leveraging this diversity, MoMaGen can train successful imitation learning policies from a single source demonstration, and these policies can be fine-tuned with as few as 40 real-world demonstrations to achieve deployment on physical robotic hardware. More details are available at our project page: momagen.github.io.

Figures (13)

• Figure 1: (left) MoMaGen uses a single human-collected demonstration to generate a large set of demonstrations, formulating data generation as a constrained optimization problem. (top-left) shows a human-collected demo for cleaning a pan with a scrub. (top-right) shows three novel object configurations with aggressive object pose randomization and additional distractors/obstacles. MoMaGen can generate novel trajectories in these diverse scenarios. (bottom-left) shows three robot base poses and (bottom-right) shows two arm trajectories for picking up the pan.
• Figure 2: MoMaGen method. Given a single source demonstration, as well as annotations for object-centric subtasks for each end-effector, MoMaGen first randomizes scene configuration, and transforms the end-effector poses from the source demo to the new objects' frame of reference. For each subtask, it tries to sample a valid base pose that satisfies reachability and visibility constraints. Once found, it plans a base and torso trajectory to reach the desired base and head camera pose while trying to look at the target object during navigation. Once arrived, it plans an arm trajectory to the pregrasp pose and uses task space control for replay, before retracting back to a tucked, neutral pose.
• Figure 3: Task visualization. Our multi-step tasks include long-range navigation, sequential and coordinated bimanual manipulation, requiring pick-and-place and contact-rich motion.
• Figure 4: Generated data diversity analysis for Tidy Table task (50 trajectories, subsampled). Given the same object randomization (D0) (a), compared to SkillMimicGen, MoMaGen samples diverse base poses (b), and as a result, diverse end-effector poses (c) and joint positions (d). MoMaGen is also the only method that can generate data for D1 randomization (red) for even greater diversity.
• Figure 5: Object visibility analysis for MoMaGen and ablations. The x-axis is the % of frames where the target object is visible during navigation, and the y-axis is the trajectory count (out of 1000). MoMaGen significantly outperforms ablations thanks to both hard and soft visibility constraints.