Challenges and Opportunities for Large-Scale Exploration with Air-Ground Teams using Semantics

TL;DR

Large-scale exploration in unknown, potentially hazardous environments is enhanced by heterogeneous air-ground robot teams. The paper introduces semantics as a lingua franca and fully opportunistic communications to coordinate onboard mapping, planning, and traversability, demonstrated in real-world and simulation settings with areas up to $157000\,m^2$ ($15.7\,\text{ha}$). Key contributions include a complete system architecture linking aerial and ground platforms, semantic mapping and panorama-based autonomy, a gossip-based data mule communication layer, and practical lessons with open-source code. The approach reduces reliance on continuous connectivity and improves robustness and scalability for applications such as search, inspection, and disaster response.

Abstract

One common and desirable application of robots is exploring potentially hazardous and unstructured environments. Air-ground collaboration offers a synergistic approach to addressing such exploration challenges. In this paper, we demonstrate a system for large-scale exploration using a team of aerial and ground robots. Our system uses semantics as lingua franca, and relies on fully opportunistic communications. We highlight the unique challenges from this approach, explain our system architecture and showcase lessons learned during our experiments. All our code is open-source, encouraging researchers to use it and build upon.

Figures (5)

• Figure 1: Top: the environments where we performed exploration were as big as $157000\,m^2$ ($15.7\,\text{ha}$) including urban (left) and rural settings (right). Bottom Left: high-altitude used to perform online mapping and communications. Bottom Right: Jackal robots using during the experiment. The LiDAR was used for navigation, state estimation, and obstacle avoidance. Cameras were used only for data logging purposes. Figures from cladera2023enabling.
• Figure 2: System architecture. Sensors are shown in orange and visualization outputs in green. Figure from miller2024spomp.
• Figure 3: outputs computed onboard the : orthomap, elevation map, and semantic map. Figure from miller2022stronger.
• Figure 4: Analysis of real-world experiments from the point-of-view. Most goals were visited during the first half of the experiment, with some goals being difficult to visit. Only two goals were not visited. Goals were roughly evenly distributed between robots, and robots spent more than 75% of the time navigating to a goal, on average. Figure from miller2024spomp.
• Figure 5: Traversability failure example. The ground robot plans a path across grass, but vegetation and tree leaves hide a pitfall. The got stuck, ending its mission.