Adaptive Obstacle-Aware Task Assignment and Planning for Heterogeneous Robot Teaming

TL;DR

This paper tackles the scalable task assignment and planning problem for heterogeneous robot teams operating in obstacle-rich environments. It introduces OATH, a framework that combines an adaptive obstacle-aware Halton sampling map, a hierarchical cluster–auction–selection allocation, and MILP-based intra-cluster task sequencing, all integrated with an LLM-based translator for real-time human guidance. Key contributions include the adaptive Halton map with obstacle-aware density, obstacle-aware clustering coupled with a cluster-weighted auction, and real-time LLM-guided replanning, validated in NVIDIA Isaac Sim and showing improved task allocation quality, scalability, and execution performance over state-of-the-art MATP baselines. The work demonstrates practical implications for logistics, inspection, and search-and-rescue by enabling scalable, interactive, and adaptable multi-robot coordination in complex environments, with potential extensions to physical deployments and learning-based enhancements.

Abstract

Multi-Agent Task Assignment and Planning (MATP) has attracted growing attention but remains challenging in terms of scalability, spatial reasoning, and adaptability in obstacle-rich environments. To address these challenges, we propose OATH: Adaptive Obstacle-Aware Task Assignment and Planning for Heterogeneous Robot Teaming, which advances MATP by introducing a novel obstacle-aware strategy for task assignment. First, we develop an adaptive Halton sequence map, the first known application of Halton sampling with obstacle-aware adaptation in MATP, which adjusts sampling density based on obstacle distribution. Second, we propose a cluster-auction-selection framework that integrates obstacle-aware clustering with weighted auctions and intra-cluster task selection. These mechanisms jointly enable effective coordination among heterogeneous robots while maintaining scalability and near-optimal allocation performance. In addition, our framework leverages an LLM to interpret human instructions and directly guide the planner in real time. We validate OATH in NVIDIA Isaac Sim, showing substantial improvements in task assignment quality, scalability, adaptability to dynamic changes, and overall execution performance compared to state-of-the-art MATP baselines. A project website is available at https://llm-oath.github.io/.

Figures (11)

• Figure 1: Problem setup in the Isaac simulation environment. A heterogeneous robot team of ground robots and drones is deployed in a maze-like environment. Two types of tasks are represented by blue and red markers, while delivery rooms are labeled B–E. The environment contains both known obstacles (walls) and initially unknown obstacles (iron gates and bushes).
• Figure 2: Overview of the OATH framework: the planning module is composed of three parts — environment and map preprocessing, task assignment, and path planning. Task assignment and path planning operate in a closed loop, where robot plans are iteratively updated as tasks are completed. At the high level, an LLM continuously interprets human instructions, providing semantic guidance to ensure obstacle-aware and adaptive task assignment throughout the process.
• Figure 3: Spatial modeling and task clustering process. The left figure shows the construction of the adaptive Halton sequence, generating obstacle-aware sampling points. The middle figure presents the Dijkstra result computed over the Halton map, producing a task-to-task distance matrix. The right figure visualizes the final clustering result: four clusters are generated using agglomerative clustering mullner2011modern based on the distance matrix, where each color represents one cluster. Notably, the clustering respects the environmental complexity, avoiding grouping tasks across walls or obstacle-dense regions.
• Figure 4: Illustration of the hierarchical task-assignment framework. Tasks are clustered using spatial proximity and obstacle-aware distances. A cluster-level auction assigns task groups to robots based on robot capabilities and the cluster’s task-type distribution, denoted by $\gamma$. Within each assigned cluster, each robot selects tasks subject to capacity limits and task-type compatibility.
• Figure 5: Comparison of heterogeneous task assignment results under different numbers of task types. Each subfigure illustrates the final task assignment for teams operating with $|\tau|=2,3,5$, respectively. The results show that each robot strictly follows its capability matrix when selecting tasks. Even when a nearby cluster contains incompatible task types, the robot does not choose it but instead travels to a more distant cluster whose tasks match its capabilities (e.g., in the $|\tau|=5$ case, Robot 1 selects a farther cluster that aligns with its feasible task set).

Theorems & Definitions (5)

• Remark 1
• Remark 2
• Remark 2
• Remark 3
• Remark 4