Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compact Multi-Object Placement Using Adjacency-Aware Reinforcement Learning

Benedikt Kreis, Nils Dengler, Jorge de Heuvel, Rohit Menon, Hamsa Perur, Maren Bennewitz

TL;DR

The paper tackles compact placement of irregular objects under adjacency constraints using reinforcement learning to control end-effector motions. It introduces adjacency-aware object neighbor concepts, a detailed RL task and architecture based on Truncated Quantile Critics with a customized reward that balances movement efficiency, spatial compactness, and collision avoidance. The approach outperforms two non-RL baselines, achieving a significantly more compact assembly with zero collisions and robust generalization to unseen shapes, suggesting strong practical utility for dense object packing under physical constraints. The dataset, generated from fresco-like fragments, supports realistic evaluation, and the method demonstrates meaningful space savings and adaptable placement strategies suitable for industrial and reconstruction contexts.

Abstract

Close and precise placement of irregularly shaped objects requires a skilled robotic system. The manipulation of objects that have sensitive top surfaces and a fixed set of neighbors is particularly challenging. To avoid damaging the surface, the robot has to grasp them from the side, and during placement, it has to maintain the spatial relations with adjacent objects, while considering the physical gripper extent. In this work, we propose a framework to learn an agent based on reinforcement learning that generates end-effector motions for placing objects as closely as possible to one another. During the placement, our agent considers the spatial constraints with neighbors defined in a given layout of the objects while avoiding collisions. Our approach learns to place compact object assemblies without the need for predefined spacing between objects, as required by traditional methods. We thoroughly evaluated our approach using a two-finger gripper mounted on a robotic arm with six degrees of freedom. The results demonstrate that our agent significantly outperforms two baseline approaches in object assembly compactness, thereby reducing the space required to position the objects while adhering to specified spatial constraints.

Compact Multi-Object Placement Using Adjacency-Aware Reinforcement Learning

TL;DR

The paper tackles compact placement of irregular objects under adjacency constraints using reinforcement learning to control end-effector motions. It introduces adjacency-aware object neighbor concepts, a detailed RL task and architecture based on Truncated Quantile Critics with a customized reward that balances movement efficiency, spatial compactness, and collision avoidance. The approach outperforms two non-RL baselines, achieving a significantly more compact assembly with zero collisions and robust generalization to unseen shapes, suggesting strong practical utility for dense object packing under physical constraints. The dataset, generated from fresco-like fragments, supports realistic evaluation, and the method demonstrates meaningful space savings and adaptable placement strategies suitable for industrial and reconstruction contexts.

Abstract

Close and precise placement of irregularly shaped objects requires a skilled robotic system. The manipulation of objects that have sensitive top surfaces and a fixed set of neighbors is particularly challenging. To avoid damaging the surface, the robot has to grasp them from the side, and during placement, it has to maintain the spatial relations with adjacent objects, while considering the physical gripper extent. In this work, we propose a framework to learn an agent based on reinforcement learning that generates end-effector motions for placing objects as closely as possible to one another. During the placement, our agent considers the spatial constraints with neighbors defined in a given layout of the objects while avoiding collisions. Our approach learns to place compact object assemblies without the need for predefined spacing between objects, as required by traditional methods. We thoroughly evaluated our approach using a two-finger gripper mounted on a robotic arm with six degrees of freedom. The results demonstrate that our agent significantly outperforms two baseline approaches in object assembly compactness, thereby reducing the space required to position the objects while adhering to specified spatial constraints.
Paper Structure (14 sections, 6 equations, 4 figures, 4 tables)

This paper contains 14 sections, 6 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Definition
  4. Our Approach
  5. Object Neighbor Concepts
  6. RL Task Description
  7. RL Architecture
  8. Reward
  9. Dataset Generation
  10. Experimental Evaluation
  11. Experimental Setup
  12. Ablation and Baselines
  13. Experimental Results
  14. Conclusion

Figures (4)

  • Figure 1: The goal of our approach is to place all objects as compactly as possible to one another while considering the objects' adjacency constraints.
  • Figure 2: Overview of our approach: a) To determine the assembly sequence, we slide a virtual window with a fixed size in a snake-like pattern (red line) over the layout. Furthermore, we determine the corresponding points by searching for the two most distant points between all shared object edges in the layout. b) The exemplary environment depicts two objects from a top-down view. The bottom one, the robot has already placed along the reference lines and it has grasped the next one in the assembly sequence, which is now between the gripper fingers. To maneuver the grasped object close to its neighbors, the RL agent uses the corresponding points to calculate the object and the reference constraints. c) At each time step $t$, the RL agent observes the state of the environment $s_t$ and sends action commands $a_t$ to the EE.
  • Figure 3: a) We attach the required operational space of the gripper to the grasped object's shape and create their union. By inflating the union by 1mm, we add a safety margin for the placement. The resulting shape is the gripper footprint. b) Baseline 2 uses the gripper footprint to find a suitable placement location by incrementally moving the shapes of the placed object and the gripper footprint until they no longer overlap. The movement is defined by the movement vector $\vec{m}$.
  • Figure 4: The plots show all results of the qualitative and quantitative evaluation. a-d) The 2D top-down view shows fresco assemblies abstracted as polygons. a) Using OUR approach results in the most compact assembly. b) Without the reference lines in OUR-ABL, there is a clear object shift and assembly skew, highlighting the necessity of their use. c) Scaling the fresco according to BL1 results in a larger bounding box due to the equal spacing. d) Placing the objects relative to each other according to BL2 results in a higher displacement compared to OUR approach, but lower than BL1. e-g) The plots show the results of the quantitative evaluation. e) OUR approach has the smallest bounding box increase (BBI-I) compared to both baselines. f) The angle difference of both baselines is smaller than OUR approach. g) OUR approach achieves the smallest mean object distance with no collisions. The collision rate of BL1 decreases with an increasing scaling factor $\alpha_\text{b}$.