SpikeSMOKE: Spiking Neural Networks for Monocular 3D Object Detection with Cross-Scale Gated Coding

TL;DR

This work applies SNNs to monocular 3D object detection and proposes the SpikeSMOKE architecture, which represents a new attempt at low-power monocular 3D object detection and presents a novel light-weight residual block that can maintain spiking computing paradigm and the highest possible detection performance.

Abstract

With the wide application of 3D object detection in some fields such as autonomous driving, its energy consumption is constantly increasing, making the research on low-power consumption alternatives a key research area. The spiking neural networks (SNNs), possessing low-power consumption characteristics, offer a novel solution for this research. Consequently, we apply SNNs to monocular 3D object detection and propose the SpikeSMOKE architecture, which represents a new attempt at low-power monocular 3D object detection. It's well known that the discrete signals of SNNs can lead to information loss compared to artificial neural networks (ANNs), which limits their feature representation capabilities. To solve this problem, inspired by the synaptic filtering mechanism of biological neurons, we propose a new Cross-Scale Gating Coding Mechanism (CSGC), which can enhance feature representation by combining cross-scale fusion of attentional methods and gated filtering mechanisms. In addition, to reduce the computation and accelerate training, we present a novel light-weight residual block that can maintain spiking computing paradigm and the highest possible detection performance. Our method is effective on the KITTI, NuScenes-mini and CIFAR10/100 datasets. Compared to the baseline SpikeSMOKE under the 3D Object Detection, the proposed SpikeSMOKE with CSGC can achieve 11.78 (+2.82, Easy), 10.69 (+3.2, Moderate), and 10.48 (+3.17, Hard) on the KITTI autonomous driving dataset by AP|R11 at 0.7 IoU threshold, respectively. It is worth noting that the results of SpikeSMOKE can significantly reduce energy consumption compared with the results of SMOKE. And SpikeSMOKE-L (lightweight) can further reduce the amount of parameters by 3 times and computation by 10 times compared to SMOKE.

Figures (5)

• Figure 1: SpikeSMOKE overall network architecture. The input is an image and the output is the 3D bounding box of the object detection and the object category. A CSGC coding mechanism is introduced, based on a cross-scale attention fusion, to reduce the performance loss due to data transformation and enhance the representation of complex features.
• Figure 2: Gated coding is different from direct coding. The proposed gated coding mechanism utilizes a cross-scale fusion module that incorporates a gating unit (CSGC) to mimic the filtering mechanism of biological neuronal synapses to produce a filtering effect on the input feature map.
• Figure 3: Light-weight residual blocks. The rightmost figure uses depth-wise convolution and point-wise convolution to replace vanilla convolution.
• Figure 4: Ablation experiments in CSGC with different attention modules at different time steps.
• Figure 5: The visualization results from monocular 3D object detection on the KITTI dataset present a clear and intuitive representation of the performance and accuracy of the detection algorithm in identifying and localizing 3D objects within the complex road scenarios depicted in the dataset.