Fusion4CA: Boosting 3D Object Detection via Comprehensive Image Exploitation

TL;DR

Fusion4CA is proposed, which is built upon the classic BEVFusion framework and dedicated to fully exploiting visual input with plug-and-play components and achieves 69.7% mAP with only 6 training epochs and a mere 3.48% increase in inference parameters.

Abstract

Nowadays, an increasing number of works fuse LiDAR and RGB data in the bird's-eye view (BEV) space for 3D object detection in autonomous driving systems. However, existing methods suffer from over-reliance on the LiDAR branch, with insufficient exploration of RGB information. To tackle this issue, we propose Fusion4CA, which is built upon the classic BEVFusion framework and dedicated to fully exploiting visual input with plug-and-play components. Specifically, a contrastive alignment module is designed to calibrate image features with 3D geometry, and a camera auxiliary branch is introduced to mine RGB information sufficiently during training. For further performance enhancement, we leverage an off-the-shelf cognitive adapter to make the most of pretrained image weights, and integrate a standard coordinate attention module into the fusion stage as a supplementary boost. Experiments on the nuScenes dataset demonstrate that our method achieves 69.7% mAP with only 6 training epochs and a mere 3.48% increase in inference parameters, yielding a 1.2% improvement over the baseline which is fully trained for 20 epochs. Extensive experiments in a simulated lunar environment further validate the effectiveness and generalization of our method. Our code will be released through Fusion4CA.

Figures (7)

• Figure 1: Key components of our Fusion4CA framework, consisting of Contrastive Alignment Module, Camera Auxiliary Branch, Cognitive Adapter and Coordinate Attention Module. Our model outperforms BEVFusion by 5% mAP at six epochs and surpasses its 20-epoch counterpart by 1.2% mAP.
• Figure 2: An overview of the proposed Fusion4CA network with four plug-and-play enhancements for visual exploitation. (1) A Contrastive Alignment Module is designed to align image features with projected point cloud features. (2) A Camera Auxiliary Branch is proposed to provide extra supervision for direct optimization of the camera branch. (3) An off-the-shelf Cognitive Adapter is inserted into the Swin Transformer while keeping its original weights frozen. (4) A standard Coordinate Attention Module is appended after convolutional fusion to capture discriminative information effectively. Note that residual connections are omitted for brevity.
• Figure 3: Illustration of the Camera Auxiliary Branch, comprising stacked residual blocks, FPN, and CenterPoint Head. The primary function is to provide supplementary supervision signals to directly optimize the camera branch.
• Figure 4: The Cognitive Adapter is inserted after the self-attention and feed-forward layers in each Swin-T block, where adaptive layer normalization, depthwise convolution and residual connections are employed to boost feature expressiveness.
• Figure 5: Illustration of Coordinate Attention Module. The module applies 1D global average pooling along two directions to compute direction-sensitive attention weights, then enhances the input via element-wise multiplication and a residual connection.