Coordination of multiple mobile manipulators for ordered sorting of cluttered objects

TL;DR

This work tackles coordinating multiple mobile manipulators to sort objects in clutter according to a prescribed order, a $\,mathcal{NP}$-hard nonmonotone rearrangement problem. It introduces a two-stage approach: (i) a search-based planner that builds a minimum-length sorting sequence $\mathcal{O}_S$ while accommodating occlusions, and (ii) a greedy multi-robot task allocator that dispatches objects to $M$ robots for parallel execution; four search variants (BFS, DFS, Best-First, and $A^*$) are analyzed for completeness and optimality where applicable. The authors provide admissible heuristics, polynomial-time utilities for accessibility and relocation decisions, and formal results (Theorems 4.2, 4.3, and Proposition 4.4) to support the method's guarantees. Empirical results on random instances and dynamic Unity-ROS simulations show that Best-First and DFS offer practical speed and robustness, achieving fast sequence generation and effective multi-robot coordination, with high success rates and near-optimal sorting sequences. The framework lays a foundation for scalable, real-world multi-robot ordered sorting in clutter and could be extended to more complex object geometries and additional constraints.

Abstract

We present a coordination method for multiple mobile manipulators to sort objects in clutter. We consider the object rearrangement problem in which the objects must be sorted into different groups in a particular order. In clutter, the order constraints could not be easily satisfied since some objects occlude other objects so the occluded ones are not directly accessible to the robots. Those objects occluding others need to be moved more than once to make the occluded objects accessible. Such rearrangement problems fall into the class of nonmonotone rearrangement problems which are computationally intractable. While the nonmonotone problems with order constraints are harder, involving with multiple robots requires another computation for task allocation. The proposed method first finds a sequence of objects to be sorted using a search such that the order constraint in each group is satisfied. The search can solve nonmonotone instances that require temporal relocation of some objects to access the next object to be sorted. Once a complete sorting sequence is found, the objects in the sequence are assigned to multiple mobile manipulators using a greedy allocation method. We develop four versions of the method with different search strategies. In the experiments, we show that our method can find a sorting sequence quickly (e.g., 4.6 sec with 20 objects sorted into five groups) even though the solved instances include hard nonmonotone ones. The extensive tests and the experiments in simulation show the ability of the method to solve the real-world sorting problem using multiple mobile manipulators.

Figures (7)

• Figure 1: Examples of tasks that should consider the order and type of objects to be manipulated. (Left) Boxes that need to be sorted into groups while the stacking order of objects is important. (Right) Parts or packages that need to be sent to the different destinations in a specific order.
• Figure 2: Examples of problem instances. (a) Seven objects are sorted into three groups $\mathcal{G}_1, \cdots, \mathcal{G}_3$. The objects have specific orders to be sorted. (b) In dense clutter, robots would not access all objects directly. The object that must be sorted in the beginning (i.e., $r_1, g_1, b_1$) are occluded.
• Figure 3: Example scenes of the experiments in simulation for $K = 5$ (objects with five colors) and varying $N$. The left picture shows the mobile manipulators (UR5 and Jackal) that manipulate objects using a suction gripper.
• Figure 4: Example complete search trees where the leaves have the same depth
• Figure 5: The computation time of Alg. \ref{['alg:sorting']} to generate a sorting sequence. Since BFS and A$^*$ are prohibitively slow with $N > 10$, we only show the results of two fast versions of Alg. \ref{['alg:sorting']}.