Collective PV-RCNN: A Novel Fusion Technique using Collective Detections for Enhanced Local LiDAR-Based Perception

TL;DR

This paper tackles the limited local LiDAR perception of autonomous vehicles due to occlusions and environmental conditions by leveraging Collective Perception. It introduces CPV-RCNN, which extends PV-RCNN++ to fuse collective detections inside the local detection pipeline, via four independent methods: Point Decoration, Collective Proposals, Raw Box Features, and Collective Voxel Set Abstraction. The study shows that combining these methods—especially CVSA with CPr—and optionally performing a late fusion step yields substantial improvements in 3D detection performance, achieving AP@IoU$_{0.7}$ up to approximately 85.5% and AP@IoU$_{0.5}$ up to about 98.6% on a synthetic CARLA/RESIST dataset. The results illustrate that integrating cooperative detections into the local detector can capture benefits of both early and late fusion while maintaining manageable communication bandwidth, with future work focusing on realistic communication models and more diverse scenarios.

Abstract

Comprehensive perception of the environment is crucial for the safe operation of autonomous vehicles. However, the perception capabilities of autonomous vehicles are limited due to occlusions, limited sensor ranges, or environmental influences. Collective Perception (CP) aims to mitigate these problems by enabling the exchange of information between vehicles. A major challenge in CP is the fusion of the exchanged information. Due to the enormous bandwidth requirement of early fusion approaches and the interchangeability issues of intermediate fusion approaches, only the late fusion of shared detections is practical. Current late fusion approaches neglect valuable information for local detection, this is why we propose a novel fusion method to fuse the detections of cooperative vehicles within the local LiDAR-based detection pipeline. Therefore, we present Collective PV-RCNN (CPV-RCNN), which extends the PV-RCNN++ framework to fuse collective detections. Code is available at https://github.com/ekut-es

Figures (1)

• Figure 1: The overall architecture of the extended PV-RCNN++ shi2023pv detector. We propose four independent extensions to the PV-RCNN++ detector, which can be used individually or in combination. ① Point Decoration (PD): The input points are decorated with additional point features, determined by the collective detections, in which the points are located. ② Collective Proposals (CPr): The collective detections are used as additional region proposals, either as additional input to the sectorized proposal centric keypoint sampling module (CPr SPC) or the RoI-grid pooling module (CPr RoI-grid). ③ Raw Box Features (RBF): The raw bounding boxes of the collective detections are used as additional input to the voxel set abstraction module. ④ Collective Voxel Set Abstraction (CVSA): A second keypoint sampling module is added, which samples keypoints within the collective detection, followed by a second voxel set abstraction module. Figure adapted and extended from shi2023pv licensed under a Creative Commons Attribution 4.0 International License.