Select2Col: Leveraging Spatial-Temporal Importance of Semantic Information for Efficient Collaborative Perception

TL;DR

Select2Col addresses the limitation of prior collaborative perception approaches that rely solely on spatial information by introducing an IoSI framework that jointly considers spatial and temporal importance of semantic data. It combines a lightweight GNN-based collaborator selector with HPHA, a fusion mechanism that incorporates multi-scale spatial attention and short-term temporal attention to produce IoSI-consistent fusion weights. Across three open datasets OPV2V, V2XSet, and V2V4Real, Select2Col demonstrates robust and substantial improvements in AP over state-of-the-art methods, while maintaining low latency and parameter counts. The approach offers a practical path toward more reliable and scalable collaborative perception in IoV, with potential extensions to IoU-aware weighting and privacy-protective collaboration.

Abstract

Collaborative perception by leveraging the shared semantic information plays a crucial role in overcoming the individual limitations of isolated agents. However, existing collaborative perception methods tend to focus solely on the spatial features of semantic information, while neglecting the importance of the temporal dimension. Consequently, the potential benefits of collaboration remain underutilized. In this article, we propose Select2Col, a novel collaborative perception framework that takes into account the \underline{s}patial-t\underline{e}mpora\underline{l} importanc\underline{e} of semanti\underline{c} informa\underline{t}ion. Within the Select2Col, we develop a collaborator selection method that utilizes a lightweight graph neural network (GNN) to estimate the importance of semantic information (IoSI) of each collaborator in enhancing perception performance, thereby identifying contributive collaborators while excluding those that potentially bring negative impact. Moreover, we present a semantic information fusion algorithm called HPHA (historical prior hybrid attention), which integrates multi-scale attention and short-term attention modules to capture the IoSI in feature representation from the spatial and temporal dimensions respectively, and assigns IoSI-consistent weights for efficient fusion of information from selected collaborators. Extensive experiments on three open datasets demonstrate that our proposed Select2Col significantly improves the perception performance compared to state-of-the-art approaches. The code associated with this research is publicly available at https://github.com/huangqzj/Select2Col/.

Figures (11)

• Figure 1: The limitation of single-agent perception. From the perspective of the ego vehicle, Vehicle A is occluded, causing a dangerous collision risk when both vehicles accelerate. Such collisions occur frequently.
• Figure 2: The overview of our proposed Select2Col framework.
• Figure 3: Diagram of our collaborator selection method.
• Figure 4: Diagram of our HPHA fusion algorithm.
• Figure 5: The comparison of computing efficiency.