Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GOPLA: Generalizable Object Placement Learning via Synthetic Augmentation of Human Arrangement

Yao Zhong, Hanzhi Chen, Simon Schaefer, Anran Zhang, Stefan Leutenegger

TL;DR

GOPLA addresses the challenge of generalizable object placement by jointly considering user preferences and 3D spatial constraints. It uses a three-stage hierarchical pipeline: a high-level multimodal language model converts open-ended instructions into structured plans, a mid-level spatial mapper grounds these plans as 3D affordance maps, and a low-level diffusion planner produces physically plausible placement poses guided by test-time costs. A scalable data generation pipeline expands few real demonstrations into diverse synthetic datasets via scene graphs and genetic-style augmentation, enabling training of the lower-level models. Across synthetic and real-world tasks, GOPLA outperforms baselines by a substantial margin, demonstrates strong generalization to unseen scenes and objects, and proves effective on real robotic hardware, highlighting its potential for assistive household manipulation with reduced data requirements.

Abstract

Robots are expected to serve as intelligent assistants, helping humans with everyday household organization. A central challenge in this setting is the task of object placement, which requires reasoning about both semantic preferences (e.g., common-sense object relations) and geometric feasibility (e.g., collision avoidance). We present GOPLA, a hierarchical framework that learns generalizable object placement from augmented human demonstrations. A multi-modal large language model translates human instructions and visual inputs into structured plans that specify pairwise object relationships. These plans are then converted into 3D affordance maps with geometric common sense by a spatial mapper, while a diffusion-based planner generates placement poses guided by test-time costs, considering multi-plan distributions and collision avoidance. To overcome data scarcity, we introduce a scalable pipeline that expands human placement demonstrations into diverse synthetic training data. Extensive experiments show that our approach improves placement success rates by 30.04 percentage points over the runner-up, evaluated on positioning accuracy and physical plausibility, demonstrating strong generalization across a wide range of real-world robotic placement scenarios.

GOPLA: Generalizable Object Placement Learning via Synthetic Augmentation of Human Arrangement

TL;DR

GOPLA addresses the challenge of generalizable object placement by jointly considering user preferences and 3D spatial constraints. It uses a three-stage hierarchical pipeline: a high-level multimodal language model converts open-ended instructions into structured plans, a mid-level spatial mapper grounds these plans as 3D affordance maps, and a low-level diffusion planner produces physically plausible placement poses guided by test-time costs. A scalable data generation pipeline expands few real demonstrations into diverse synthetic datasets via scene graphs and genetic-style augmentation, enabling training of the lower-level models. Across synthetic and real-world tasks, GOPLA outperforms baselines by a substantial margin, demonstrates strong generalization to unseen scenes and objects, and proves effective on real robotic hardware, highlighting its potential for assistive household manipulation with reduced data requirements.

Abstract

Robots are expected to serve as intelligent assistants, helping humans with everyday household organization. A central challenge in this setting is the task of object placement, which requires reasoning about both semantic preferences (e.g., common-sense object relations) and geometric feasibility (e.g., collision avoidance). We present GOPLA, a hierarchical framework that learns generalizable object placement from augmented human demonstrations. A multi-modal large language model translates human instructions and visual inputs into structured plans that specify pairwise object relationships. These plans are then converted into 3D affordance maps with geometric common sense by a spatial mapper, while a diffusion-based planner generates placement poses guided by test-time costs, considering multi-plan distributions and collision avoidance. To overcome data scarcity, we introduce a scalable pipeline that expands human placement demonstrations into diverse synthetic training data. Extensive experiments show that our approach improves placement success rates by 30.04 percentage points over the runner-up, evaluated on positioning accuracy and physical plausibility, demonstrating strong generalization across a wide range of real-world robotic placement scenarios.
Paper Structure (14 sections, 8 equations, 4 figures, 3 tables)

This paper contains 14 sections, 8 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Method
  4. Problem Formulation
  5. Hierarchical Object Placement Learning
  6. Scalable Placement Data Generation
  7. Placement Model Training
  8. Implementation Details
  9. Experiments
  10. Experimental Setups
  11. Results and Discussions
  12. Ablation Studies
  13. Real Robot Experiments
  14. Conclusion

Figures (4)

  • Figure 1: Overview of our proposed placement model. A hierarchical model is used to infer the pose of an object to be placed, capturing both the user preference and the physical plausibility.
  • Figure 2: Overview of our data generation pipeline and the generated data samples augmented from the real scene.
  • Figure 3: Predicted placement by ours and the baselines. Our method not only captures the preference imposed by the instruction, but respects spatial constraints as well.
  • Figure 4: Robot integrated with our model can execute versatile downstream arrangement tasks in the real world.