LLM-Grounded Dynamic Task Planning with Hierarchical Temporal Logic for Human-Aware Multi-Robot Collaboration

TL;DR

This work tackles long-horizon, human-aware multi-robot planning by grounding LLM-generated reasoning inside a hierarchical LTL framework (H-LTL_f) and executing it via a receding horizon loop. The approach combines open-vocabulary perception, hierarchical formal specifications, and dynamic task allocation to produce correct-by-construction plans in dynamic environments, with real-time replanning and safety checks. Key contributions include the hierarchical LTL formulation, product team models with explicit intra- and inter-task switches, and a receding horizon execution pipeline that handles stochastic durations and human motion. The results show improved success rates, reduced planning latency, and robust collaboration across both simulated and real-world human-robot interactions, indicating practical impact for scalable human-aware robotics.

Abstract

While Large Language Models (LLM) enable non-experts to specify open-world multi-robot tasks, the generated plans often lack kinematic feasibility and are not efficient, especially in long-horizon scenarios. Formal methods like Linear Temporal Logic (LTL) offer correctness and optimal guarantees, but are typically confined to static, offline settings and struggle with computational scalability. To bridge this gap, we propose a neuro-symbolic framework that grounds LLM reasoning into hierarchical LTL specifications and solves the corresponding Simultaneous Task Allocation and Planning (STAP) problem. Unlike static approaches, our system resolves stochastic environmental changes, such as moving users or updated instructions via a receding horizon planning (RHP) loop with real-time perception, which dynamically refines plans through a hierarchical state space. Extensive real-world experiments demonstrate that our approach significantly outperforms baseline methods in success rate and interaction fluency while minimizing planning latency.

Figures (8)

• Figure 1: A demonstration of the proposed dynamic planning with the human instruction, "Give Person A the bottle and Person B the chips." At Snapshot 1, the system detected the persons' movement, so it began planning for the possible future positions of the persons. At Snapshot 2, the positions of the persons invalidated the original plan, so the current actions stopped immediately, and the system switched to the new plan once it's ready.
• Figure 2: The proposed method flowchart. In the Hierarchical Team Models panel, we illustrate the hierarchical team models for two leaf specifications and two robots. The product team models corresponding to each leaf specification are shown inside the regions shaded in light orange and light green. Within each product team model, $\zeta_{\mathrm{in}}^1$ are drawn as purple bidirectional dashed arrows, while $\zeta_{\mathrm{in}}^2$ are drawn as pink unidirectional arrows. Inter-spec transitions $\zeta_{\mathrm{inter}}^1$ are shown as blue bidirectional arrows, and $\zeta_{\mathrm{inter}}^2$ are shown as red unidirectional arrows. The path marked in yellow highlights a specific plan.
• Figure 3: Pick planning pipeline.
• Figure 4: The proposed robot-to-robot handover method.
• Figure 5: Simulation experiment setup for 2-4-square and 2-4-line.