PVAFN: Point-Voxel Attention Fusion Network with Multi-Pooling Enhancing for 3D Object Detection

TL;DR

PVAFN tackles the semantic gap that arises when fusing LiDAR point clouds with voxel-based features by introducing a Point-Voxel Attention Fusion Module that adaptively merges point, voxel, and BEV representations. A subsequent Multi-Pooling Enhancement Module enhances RoI refinement with a RoI Clustering Pooling Head and a RoI Pyramid Pooling Head to capture both key geometric details and global context. The approach relies on attention-guided fusion and multi-scale pooling to improve robustness against sparse data and false detections, showing competitive gains on KITTI and Waymo compared to strong baselines. The work provides practical improvements for 3D object detection and declares code and models forthcoming for public use.

Abstract

The integration of point and voxel representations is becoming more common in LiDAR-based 3D object detection. However, this combination often struggles with capturing semantic information effectively. Moreover, relying solely on point features within regions of interest can lead to information loss and limitations in local feature representation. To tackle these challenges, we propose a novel two-stage 3D object detector, called Point-Voxel Attention Fusion Network (PVAFN). PVAFN leverages an attention mechanism to improve multi-modal feature fusion during the feature extraction phase. In the refinement stage, it utilizes a multi-pooling strategy to integrate both multi-scale and region-specific information effectively. The point-voxel attention mechanism adaptively combines point cloud and voxel-based Bird's-Eye-View (BEV) features, resulting in richer object representations that help to reduce false detections. Additionally, a multi-pooling enhancement module is introduced to boost the model's perception capabilities. This module employs cluster pooling and pyramid pooling techniques to efficiently capture key geometric details and fine-grained shape structures, thereby enhancing the integration of local and global features. Extensive experiments on the KITTI and Waymo datasets demonstrate that the proposed PVAFN achieves competitive performance. The code and models will be available.

Figures (5)

• Figure 1: Comparison between (b) PV-RCNN ref8 and (c) our PVAFN. Green, blue, and yellow boxes denote predicted cars, pedestrians, and cyclists, while red boxes are false detections. PVAFN effectively minimizes false detections, such as utility boxes, road signs, and walls.
• Figure 2: Overall architecture of the proposed PVAFN. First, the raw point cloud undergoes keypoint sampling and voxelization. The resulting keypoint, voxel, and BEV features are fused using the point-voxel attention fusion module, which employs self-attention, point-voxel attention, and residual connections. The multi-pooling enhancement module then extracts geometric and fine-grained features for proposal generation and refinement.
• Figure 3: Illustration of Point-Voxel Attention. It introduces a learnable gating function $\sigma_*$, composed of attention and graph operators, which can dynamically select attention components to achieve different optimization effects.
• Figure 4: Comparison of cluster pooling head and grid pooling head. For sparse and uneven RoIs, (a) (i.e., the grid pooling method) fuses all points within the RoI, including background information such as the ground. In contrast, (b) (i.e., the clustering pooling method) focuses only on the key geometric information of the target within the bounding box.
• Figure 5: Visualization results on the KITTI validation set. (a) shows the camera image, and (b) shows the corresponding front view of the point cloud. The ground truth bounding box is green, and the predicted bounding box is red.