Bandwidth-Adaptive Spatiotemporal Correspondence Identification for Collaborative Perception

TL;DR

This work tackles correspondence identification (CoID) in multi-robot collaborative perception under limited V2X bandwidth. It introduces a bandwidth-adaptive framework that builds a spatiotemporal graph for each agent, uses a heterogeneous graph attention network to fuse visual, spatial, and temporal cues, and employs a dual spatiotemporal pooling mechanism to compress information. A Top_K candidate-sharing strategy, guided by node- and graph-level similarities, enables progressive data exchange within bandwidth constraints and a circle-loss-based graph matching for final correspondences. Experiments in CARLA-SUMO CAD simulations demonstrate state-of-the-art CoID performance and data-sharing efficiency across normal and crowded traffic, with substantial improvements in covisible object retrieval and robustness to bandwidth variation.

Abstract

Correspondence identification (CoID) is an essential capability in multi-robot collaborative perception, which enables a group of robots to consistently refer to the same objects within their respective fields of view. In real-world applications, such as connected autonomous driving, vehicles face challenges in directly sharing raw observations due to limited communication bandwidth. In order to address this challenge, we propose a novel approach for bandwidth-adaptive spatiotemporal CoID in collaborative perception. This approach allows robots to progressively select partial spatiotemporal observations and share with others, while adapting to communication constraints that dynamically change over time. We evaluate our approach across various scenarios in connected autonomous driving simulations. Experimental results validate that our approach enables CoID and adapts to dynamic communication bandwidth changes. In addition, our approach achieves 8%-56% overall improvements in terms of covisible object retrieval for CoID and data sharing efficiency, which outperforms previous techniques and achieves the state-of-the-art performance. More information is available at: https://gaopeng5.github.io/acoid.

Figures (5)

• Figure 1: A motivating example of CoID under the communication bandwidth constraint for collaborative perception in connected autonomous driving. In order to enable connected vehicles to refer to the same street objects, they must share spatiotemporal information to identify object correspondences, while satisfying the bandwidth constraint.
• Figure 2: An overview of our proposed bandwidth-adaptive spatiotemporal CoID approach. A sequence of observations is represented as a spatiotemporal graph. A spatiotemporal graph attention network is used to generate node-level embeddings by integrating spatiotemporal visual cues. Then, a heterogeneous graph pooling operation is designed to produce comprehensive graph-level embeddings that explicitly encode the importance of spatial and temporal cues. Leveraging both node-level and graph-level embeddings, our approach enables the sharing of node candidates that are likely to be observed by collaborators, resulting in effective data sharing that adapts to communication bandwidth constraints.
• Figure 3: Qualitative results obtained by our approach in normal traffic scenarios (the first row) and crowd traffic scenarios (the second row), as well as comparisons with the DGMC method and the ground truth. The sequence of points denotes a history of observations, which consists of 1-5 points indicating object locations in the past 1-5 time steps. Red points denote non-covisible objects that can only be observed by Vehicle 1. Green points represent the identified covisible objects that can be observed by both Vehicle 1 (in the orange bounding box) and Vehicle 2 (in the blue bounding box). All results are computed in the setup that Vehicle 1 receives information from Vehicle 2, and Vehicle 2 aims to share covisible objects with Vehicle 1.
• Figure 4: Comparison between DGMC and our approach on the F1 score given different number of iterations and bandwidth constraints.
• Figure 5: Analysis of our approach's characteristics in the CAD simulation, including the effect of the length of temporal sequence based on BIS and the improvements of CoID based on precision.