Hierarchical Temporal Logic Task and Motion Planning for Multi-Robot Systems

TL;DR

The paper addresses multi-robot task and motion planning under hierarchical sc-LTL specifications by transforming task planning into a shortest-path problem on a product graph that couples a CS-based transition system with DFAs for leaf specifications. It leverages Graphs of Convex Sets (GCS) and an IRIS-RRT-inspired region construction to handle high-dimensional motion planning, while incorporating handover constraints via a mixed-integer convex program (MICP). The authors prove soundness and completeness under mild assumptions and demonstrate scalability and performance gains across planar, multi-robot handover, mobile-manipulator, and industrial-like scenarios, including hardware validation. The work provides an efficient, scalable framework for long-horizon, collaborative multi-robot tasks with expressive HLTL specifications and contributes open-source code for reproducibility and broader adoption.

Abstract

Task and motion planning (TAMP) for multi-robot systems, which integrates discrete task planning with continuous motion planning, remains a challenging problem in robotics. Existing TAMP approaches often struggle to scale effectively for multi-robot systems with complex specifications, leading to infeasible solutions and prolonged computation times. This work addresses the TAMP problem in multi-robot settings where tasks are specified using expressive hierarchical temporal logic and task assignments are not pre-determined. Our approach leverages the efficiency of hierarchical temporal logic specifications for task-level planning and the optimization-based graph of convex sets method for motion-level planning, integrating them within a product graph framework. At the task level, we convert hierarchical temporal logic specifications into a single graph, embedding task allocation within its edges. At the motion level, we represent the feasible motions of multiple robots through convex sets in the configuration space, guided by a sampling-based motion planner. This formulation allows us to define the TAMP problem as a shortest path search within the product graph, where efficient convex optimization techniques can be applied. We prove that our approach is both sound and complete under mild assumptions. Additionally, we extend our framework to cooperative pick-and-place tasks involving object handovers between robots. We evaluate our method across various high-dimensional multi-robot scenarios, including simulated and real-world environments with quadrupeds, robotic arms, and automated conveyor systems. Our results show that our approach outperforms existing methods in execution time and solution optimality while effectively scaling with task complexity.

Figures (14)

• Figure 1: Given the hierarchical temporal logic specifications \ref{['eq:four-robots HLTL']}, which specify transferring the yellow, blue, and red objects in order, our approach efficiently generates collision-free trajectories for four robotic manipulators, with a total of 28 degrees of freedom, to collaboratively complete the task in the shared workspace.
• Figure 2: An architecture for hierarchical temporal logic task and motion planning, where the transition system can be precomputed offline.
• Figure 3: An example of generating the labeled convex region for a multi-robot system. The atomic proposition is $\mathsf{target\ a}$. In Fig. \ref{['fig:convex_set_a']}, the atomic proposition $\pi$ is used to compute the labeled joint configuration $s_{\text{label}}$ through robot inverse kinematics. In this configuration, the bottom robot reaches the position labeled as $\mathsf{target\ a}$, while the configurations of the remaining robots are unconstrained. Using $s_{\text{label}}$ as a seed point, the IRIS-NP algorithm can generate a convex set region that contains this seed point. The sampled configurations inside the convex region are shown in Fig. \ref{['fig:convex_set_b']}. Note that not all configurations of the bottom robot ensure reaching $\mathsf{target\ a}$. Fig. \ref{['fig:convex_set_c']} illustrates the labeled convex region $S_{\text{label}}$ generated by adding the bottom robot's end-effector position constraints in IRIS-NP algorithm, ensuring that all configurations in the labeled convex region satisfy the atomic proposition $\mathsf{target\ a}$.
• Figure 4: A 2D example for collision-free convex set construction. The orange blocks represent obstacles. Given the robot's start configuration $s_{\text{start}}$ and goal configuration $s_{\text{target}}$, IRIS-RRT algorithm uses RRT path (black line) as a guide to approximate the connected convex sets to connect the start and goal configurations. In the graph, $s_{\text{seed}_i}$ represents the seed configuration anchoring the $i$-th convex set, which is depicted as a rectangle surrounded by dashed lines. Each pair of seed configuration and its corresponding convex set is highlighted in the same color. Note that the seed configurations typically lie at the intersections of the path generated by RRT and the previous convex sets. Moreover, the path generated by RRT does not need to be smooth or optimal, as it primarily serves to guide the construction of convex sets.
• Figure 5: The DFAs corresponding to specifications have their accepting states highlighted in yellow. In Fig. \ref{['fig:product']}, only the automaton states for leaf specifications are displayed since the states of non-leaf specifications can be deduced from those of the leaf specifications in a bottom-up manner. Within the diagram, the labels inside the nodes and along the edges are derived from two parts: the first component is from $\phi_{2}^{1}$ and the second from $\phi_{2}^{2}$. There are four paths leading from the initial state $(\text{init}1, \text{init}1)$ to the accepting state $(\text{accept}4, \text{accept}2)$. Notably, in all these paths, the symbol $c$ is not the last one to be fulfilled.

Theorems & Definitions (19)

• Definition 3.1: Deterministic Finite Automaton (DFA) baier2008principles
• Definition 3.2: Hierarchical sc-LTL luo2024simultaneous
• Example 1: Hierarchical sc-LTL
• Definition 3.3: Specification hierarchy tree luo2024simultaneous
• Definition 3.4: CS-based Transition System
• Definition 3.5: State-Specification Sequence luo2024simultaneous
• Definition 4.1: Trajectory
• Definition 5.1: Product DFA (PDFA)
• Definition 5.2: Total PDFA (TPDFA)
• Example \ref{exmp:h-sc-ltl}