Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Spatio-Temporal Signal Recognition on Edge Devices Using PointLCA-Net

Sanaz Mahmoodi Takaghaj

TL;DR

PointLCA-Net addresses the challenge of energy-efficient spatio-temporal signal recognition on edge devices by coupling PointNet-based feature extraction with a memory-efficient Exemplar LCA-Decoder implemented on neuromorphic hardware. The method uses a two-stage pipeline: (i) PointNet extracts robust features from spatio-temporal point clouds and builds a feature dictionary, and (ii) a single-layer spiking encoder-decoder with the Locally Competitive Algorithm encodes these features for sparse, low-power classification, with in-memory computation mapped to memristor crossbar arrays. Key contributions include the neuromorphic adaptation of PointNet/PointNet++, the first event-cloud application to NMNIST and SHD on this platform, and a demonstrable pathway for energy-efficient spatio-temporal recognition on edge devices, with potential extensions to architectures like Point Transformer. The findings show high recognition accuracy across NMNIST, SHD, and DVS128 with substantially reduced energy and computational load compared to comparable methods, highlighting practical impact for deploying advanced neural architectures in energy-constrained environments.

Abstract

Recent advancements in machine learning, particularly through deep learning architectures like PointNet, have transformed the processing of three-dimensional (3D) point clouds, significantly improving 3D object classification and segmentation tasks. While 3D point clouds provide detailed spatial information, spatio-temporal signals introduce a dynamic element that accounts for changes over time. However, applying deep learning techniques to spatio-temporal signals and deploying them on edge devices presents challenges, including real-time processing, memory capacity, and power consumption. To address these issues, this paper presents a novel approach that combines PointNet's feature extraction with the in-memory computing capabilities and energy efficiency of neuromorphic systems for spatio-temporal signal recognition. The proposed method consists of a two-stage process: in the first stage, PointNet extracts features from the spatio-temporal signals, which are then stored in non-volatile memristor crossbar arrays. In the second stage, these features are processed by a single-layer spiking neural encoder-decoder that employs the Locally Competitive Algorithm (LCA) for efficient encoding and classification. This work integrates the strengths of both PointNet and LCA, enhancing computational efficiency and energy performance on edge devices. PointLCA-Net achieves high recognition accuracy for spatio-temporal data with substantially lower energy burden during both inference and training than comparable approaches, thus advancing the deployment of advanced neural architectures in energy-constrained environments.

Efficient Spatio-Temporal Signal Recognition on Edge Devices Using PointLCA-Net

TL;DR

PointLCA-Net addresses the challenge of energy-efficient spatio-temporal signal recognition on edge devices by coupling PointNet-based feature extraction with a memory-efficient Exemplar LCA-Decoder implemented on neuromorphic hardware. The method uses a two-stage pipeline: (i) PointNet extracts robust features from spatio-temporal point clouds and builds a feature dictionary, and (ii) a single-layer spiking encoder-decoder with the Locally Competitive Algorithm encodes these features for sparse, low-power classification, with in-memory computation mapped to memristor crossbar arrays. Key contributions include the neuromorphic adaptation of PointNet/PointNet++, the first event-cloud application to NMNIST and SHD on this platform, and a demonstrable pathway for energy-efficient spatio-temporal recognition on edge devices, with potential extensions to architectures like Point Transformer. The findings show high recognition accuracy across NMNIST, SHD, and DVS128 with substantially reduced energy and computational load compared to comparable methods, highlighting practical impact for deploying advanced neural architectures in energy-constrained environments.

Abstract

Recent advancements in machine learning, particularly through deep learning architectures like PointNet, have transformed the processing of three-dimensional (3D) point clouds, significantly improving 3D object classification and segmentation tasks. While 3D point clouds provide detailed spatial information, spatio-temporal signals introduce a dynamic element that accounts for changes over time. However, applying deep learning techniques to spatio-temporal signals and deploying them on edge devices presents challenges, including real-time processing, memory capacity, and power consumption. To address these issues, this paper presents a novel approach that combines PointNet's feature extraction with the in-memory computing capabilities and energy efficiency of neuromorphic systems for spatio-temporal signal recognition. The proposed method consists of a two-stage process: in the first stage, PointNet extracts features from the spatio-temporal signals, which are then stored in non-volatile memristor crossbar arrays. In the second stage, these features are processed by a single-layer spiking neural encoder-decoder that employs the Locally Competitive Algorithm (LCA) for efficient encoding and classification. This work integrates the strengths of both PointNet and LCA, enhancing computational efficiency and energy performance on edge devices. PointLCA-Net achieves high recognition accuracy for spatio-temporal data with substantially lower energy burden during both inference and training than comparable approaches, thus advancing the deployment of advanced neural architectures in energy-constrained environments.

Paper Structure

This paper contains 16 sections, 10 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. PointLCA-Net
  4. Feature Extraction
  5. Overview of the Exemplar LCA-Decoder
  6. Experiment Setup
  7. NMNIST
  8. Spiking Heidelberg Digits (SHD)
  9. DVS128
  10. Dataset Pre-Processing
  11. Workload and Energy Efficiency Analysis
  12. Neuromorphic Hardware Mapping
  13. Evaluation
  14. Discussion
  15. Conclusion
  16. ...and 1 more sections

Figures (4)

  • Figure 1: PointLCA-Net Architecture: Features ($\phi_i$) are extracted from the training data and stored in the synaptic weights of the Exemplar LCA-Decoder. The orange arrows represent the inference process, which follows the completion of training (feature extraction).
  • Figure 2: Feature Extraction using PointNets.
  • Figure 3: Data Pre-processing
  • Figure 4: PointLCA-Net Hardware Deployment