How to Coordinate UAVs and UGVs for Efficient Mission Planning? Optimizing Energy-Constrained Cooperative Routing with a DRL Framework

TL;DR

This work tackles energy-constrained cooperative routing for heterogeneous UAV-UGV teams by integrating an encoder-decoder transformer-driven DRL framework with a sortie-based agent-switching strategy. It formalizes the problem as both a bi-level heuristic benchmark (outer UGV MSC/TSP and inner UAV E-VRPTW) and a DRL approach trained via REINFORCE, achieving superior solution quality and runtime efficiency across multiple problem sizes and configurations. The framework demonstrates robust generalization to larger scales, alternative team mixes, and diverse task-point distributions, and it supports online replanning in dynamic scenarios. Overall, the approach offers a scalable, robust, and practical solution for coordinating aerial and ground agents in mission planning, with clear applicability to disaster response and environmental monitoring.

Abstract

Efficient mission planning for cooperative systems involving Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) requires addressing energy constraints, scalability, and coordination challenges between agents. UAVs excel in rapidly covering large areas but are constrained by limited battery life, while UGVs, with their extended operational range and capability to serve as mobile recharging stations, are hindered by slower speeds. This heterogeneity makes coordination between UAVs and UGVs critical for achieving optimal mission outcomes. In this work, we propose a scalable deep reinforcement learning (DRL) framework to address the energy-constrained cooperative routing problem for multi-agent UAV-UGV teams, aiming to visit a set of task points in minimal time with UAVs relying on UGVs for recharging during the mission. The framework incorporates sortie-wise agent switching to efficiently manage multiple agents, by allocating task points and coordinating actions. Using an encoder-decoder transformer architecture, it optimizes routes and recharging rendezvous for the UAV-UGV team in the task scenario. Extensive computational experiments demonstrate the framework's superior performance over heuristic methods and a DRL baseline, delivering significant improvements in solution quality and runtime efficiency across diverse scenarios. Generalization studies validate its robustness, while dynamic scenario highlights its adaptability to real-time changes with a case study. This work advances UAV-UGV cooperative routing by providing a scalable, efficient, and robust solution for multi-agent mission planning.

Figures (12)

• Figure 1: Illustration of the fuel-constrained UAV-UGV cooperative routing problem. UAV-specific task points (red hollow circles) and ground task points (blue solid circles) must be visited, with rendezvous points (black circles) for UAVs recharging on the UGVs. The goal is to plan UAV and UGV routes (red dashed and blue solid paths, respectively) to minimize total mission time while adhering to UAV fuel and UGV speed constraints.
• Figure 1: Refuel stops obtained from MSC, with circles indicating the fuel coverage radius of the UAV. (a) Subproblem 1: starting depot (Refuel Stop 1) with allocated UAV points. (b) Subproblem 2: Refuel Stop 1 as the origin and Refuel Stop 2 as the destination with allocated UAV points. (c) Subproblem 3: Refuel Stop 2 as the origin and Refuel Stop 3 as the destination with allocated UAV points. (d) Subproblem 4: Refuel Stop 3 as the origin and Refuel Stop 4 as the destination with allocated UAV points.
• Figure 2: UAV route sorties as obtained after solving E-VRPTW. (a) UAV route sorties for Subproblem 1. (b) UAV and UGV route sorties for Subproblem 2. (c) UAV and UGV route sorties for Subproblem 3. (d) UAV and UGV route sorties for Subproblem 4.
• Figure 3: Architecture of the proposed transformer network. The encoder consists of three attention layers that process raw input data to generate node embeddings. The decoder employs an agent selection strategy to determine the active agent type (UAV or UGV) and constructs a context vector based on the current state of the selected agent. It uses the input embeddings and the context vector to determine action for the selected active agent.
• Figure 4: Agent selection strategy during the mission progress