Placeit! A Framework for Learning Robot Object Placement Skills

TL;DR

This work presents Placeit!, a versatile framework that automatically generates diverse, valid object placement poses in simulation using quality-diversity optimization. By modeling placements as interactions between objects and supports and validating them via gravity and contact dynamics, Placeit! produces a rich, robust dataset for training simulation-based robotics models and enables effective sim-to-real transfer through a domain-randomized QD-based pick-and-place pipeline (QDGP). The approach outperforms prior, priors-based sampling methods across multiple scenarios and demonstrates approximately a 90% real-world success rate over 120 deployments, underscoring its potential as a data-generation engine for open-environment manipulation and foundation-model training. Limitations include dependencies on simulation fidelity and current focus on rigid objects, with future work aiming to extend to dense scenes and deformable/ articulated objects and to integrate with reinforcement learning for end-to-end manipulation policies.

Abstract

Robotics research has made significant strides in learning, yet mastering basic skills like object placement remains a fundamental challenge. A key bottleneck is the acquisition of large-scale, high-quality data, which is often a manual and laborious process. Inspired by Graspit!, a foundational work that used simulation to automatically generate dexterous grasp poses, we introduce Placeit!, an evolutionary-computation framework for generating valid placement positions for rigid objects. Placeit! is highly versatile, supporting tasks from placing objects on tables to stacking and inserting them. Our experiments show that by leveraging quality-diversity optimization, Placeit! significantly outperforms state-of-the-art methods across all scenarios for generating diverse valid poses. A pick&place pipeline built on our framework achieved a 90% success rate over 120 real-world deployments. This work positions Placeit! as a powerful tool for open-environment pick-and-place tasks and as a valuable engine for generating the data needed to train simulation-based foundation models in robotics.

Figures (8)

• Figure 1: Placeit! takes two object meshes as input and automatically finds diverse ways for them to interact. The resulting poses can be applied to a wide variety of scenarios, enabling robust robotic manipulations.
• Figure 2: Placeit! principle. The framework uses QD optimization to explore the interactions between an object and a support. A mutation-selection process efficiently explores the space of possible placement solutions from a parameter space.
• Figure 3: QDGP pipeline. Grasp poses are generated using QDG huber2024speeding, and placing poses with Placeit!. The combined poses are validated before being deployed on the physical robot to perform the full pick-and-place sequence.
• Figure 4: Example solutions. The top row shows solutions randomly sampled from a CMA_MAE run. The bottom row shows robust solutions from the same run, identified using our domain randomization criterion.
• Figure 5: Comparison between methods. This figure shows the evolution of the diversity of placement solutions found ($cvg(\Phi^s)$) throughout the optimization process for each simulation scenario. The CMA_MAE method without priors significantly outperforms all other methods across most scenarios.