MCOP: Multi-UAV Collaborative Occupancy Prediction

TL;DR

MCOP tackles the limitations of Bird's-Eye-View based multi-UAV perception by delivering a 3D occupancy-aware framework that preserves geometry and semantics while curbing communications. It introduces four modules—Spatial-Aware Feature Encoder, Altitude-Aware Reduction, Dual-Mask Perceptual Guidance, and Cross-Agent Feature Integration—to enable efficient cross-UAV fusion and accurate 3D occupancy segmentation. The approach achieves state-of-the-art mIoU with dramatically reduced data transmission (as low as 0.23 MB per transmission) across multiple occupancy datasets extended for UAV use, illustrating robust performance under occlusion and high altitude. By enriching three datasets with occupancy annotations and proposing a joint optimization objective, MCOP advances practical, scalable collaborative perception for UAV swarms in real-time applications.

Abstract

Unmanned Aerial Vehicle (UAV) swarm systems necessitate efficient collaborative perception mechanisms for diverse operational scenarios. Current Bird's Eye View (BEV)-based approaches exhibit two main limitations: bounding-box representations fail to capture complete semantic and geometric information of the scene, and their performance significantly degrades when encountering undefined or occluded objects. To address these limitations, we propose a novel multi-UAV collaborative occupancy prediction framework. Our framework effectively preserves 3D spatial structures and semantics through integrating a Spatial-Aware Feature Encoder and Cross-Agent Feature Integration. To enhance efficiency, we further introduce Altitude-Aware Feature Reduction to compactly represent scene information, along with a Dual-Mask Perceptual Guidance mechanism to adaptively select features and reduce communication overhead. Due to the absence of suitable benchmark datasets, we extend three datasets for evaluation: two virtual datasets (Air-to-Pred-Occ and UAV3D-Occ) and one real-world dataset (GauUScene-Occ). Experiments results demonstrate that our method achieves state-of-the-art accuracy, significantly outperforming existing collaborative methods while reducing communication overhead to only a fraction of previous approaches.

Figures (4)

• Figure 1: Comparison of Multi-UAV collaborative occupancy prediction and other collaborative methods. 2D-to-2D method lacks height information and cannot effectively reconstruct the 3D details of the scene. 3D-to-3D method requires transmitting high-dimensional occupancy features, which demands high bandwidth and affects real-time performance. MCOP compresses occupancy features, balancing transmission rate and prediction quality.
• Figure 2: The overall framework of MCOP. Each UAV uses an image backbone to extract multi-scale features, which are processed by the Spatial-Aware Feature Encoder (SAFE) to generate 3D occupancy features. The Altitude-Aware Reduction (AAR) compresses these 3D features into compact 2D BEV representation for efficient communication. Dual-Mask Perceptual Guidance (DMPG) coordinates the sharing of relevant information among UAVs based on perception quality. The Cross-Agent Feature Integration (CAFI) module fuses local and received features into unified 3D representation. Finally, the Occ Head predicts 3D occupancy segmentation, resulting in comprehensive environmental perception among UAVs.
• Figure 3: Illustration of Altitude-Aware Reduction and Dual-Mask Perceptual Guidance. (a) demonstrates how a pillar in the 3D occupancy feature is compressed into a grid in the 2D BEV feature. (b) illustrates the details of DMPG, where the ego UAV first broadcasts its request mask within the network, and then the connected UAVs select and transmit high-quality features that are needed by the ego UAV.
• Figure 4: Visualization results on GauUScene-Occ. MCOP achieves better perception for distant and occluded objects.