Fast2comm:Collaborative perception combined with prior knowledge

TL;DR

Fast2comm tackles bandwidth-limited, robust cooperative perception by introducing prior knowledge into cross-agent fusion. It combines a prior-supervised confidence feature generator with GT bounding box-based feature selection and decouples training and inference fusion to adapt bandwidth. The approach achieves improved detection performance under localization errors across OPV2V, V2XSet, and DAIR-V2X, with reduced communication volume and robust fusion. The results demonstrate a practical, scalable framework for real-world V2X and multi-robot systems.

Abstract

Collaborative perception has the potential to significantly enhance perceptual accuracy through the sharing of complementary information among agents. However, real-world collaborative perception faces persistent challenges, particularly in balancing perception performance and bandwidth limitations, as well as coping with localization errors. To address these challenges, we propose Fast2comm, a prior knowledge-based collaborative perception framework. Specifically, (1)we propose a prior-supervised confidence feature generation method, that effectively distinguishes foreground from background by producing highly discriminative confidence features; (2)we propose GT Bounding Box-based spatial prior feature selection strategy to ensure that only the most informative prior-knowledge features are selected and shared, thereby minimizing background noise and optimizing bandwidth efficiency while enhancing adaptability to localization inaccuracies; (3)we decouple the feature fusion strategies between model training and testing phases, enabling dynamic bandwidth adaptation. To comprehensively validate our framework, we conduct extensive experiments on both real-world and simulated datasets. The results demonstrate the superior performance of our model and highlight the necessity of the proposed methods. Our code is available at https://github.com/Zhangzhengbin-TJ/Fast2comm.

Figures (8)

• Figure 1: The overall architecture of the proposed Fast2comm. The framework consists of six modules: Encoder, Confidence Feature Generation module, GT Bbox-Based Feature Select module, Feature Share, Feature Fusion, and Decoder. The details of each individual component are illustrated in Section \ref{['fast2comm']}.
• Figure 2: The process of the proposed attention fusion module and prior supervision.
• Figure 3: The process of the proposed 4D bounding box generating.
• Figure 4: The process of the proposed GT Bbox-Based feature generator.
• Figure 5: Collaborative perception performance comparison with varying communication volume.