Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Energy-Efficient Spiking Recurrent Neural Network for Gesture Recognition on Embedded GPUs

Marzieh Hassanshahi Varposhti, Mahyar Shahsavari, Marcel van Gerven

TL;DR

The paper tackles gesture recognition on energy-constrained edge devices by deploying a spiking recurrent neural network with liquid time constant neurons on embedded GPUs. It leverages event-based DVS data, PyTorch-based training with forward propagation through time (FPTT), and a four-layer LTC-SRNN to balance temporal accuracy with power efficiency. Key contributions include a comprehensive cross-device benchmark showing Jetson Nano achieving strong energy efficiency relative to a high-end GPU and demonstrating that batch processing can boost frame rates without sacrificing accuracy, along with a detailed inference workflow for edge deployment. The work has practical significance for real-time human-computer interaction on portable devices, enabling robust temporal-spatial interpretation with low energy budgets.

Abstract

Implementing AI algorithms on event-based embedded devices enables real-time processing of data, minimizes latency, and enhances power efficiency in edge computing. This research explores the deployment of a spiking recurrent neural network (SRNN) with liquid time constant neurons for gesture recognition. We focus on the energy efficiency and computational efficacy of NVIDIA Jetson Nano embedded GPU platforms. The embedded GPU showcases a 14-fold increase in power efficiency relative to a conventional GPU, making a compelling argument for its use in energy-constrained applications. The study's empirical findings also highlight that batch processing significantly boosts frame rates across various batch sizes while maintaining accuracy levels well above the baseline. These insights validate the SRNN with liquid time constant neurons as a robust model for interpreting temporal-spatial data in gesture recognition, striking a critical balance between processing speed and power frugality.

Energy-Efficient Spiking Recurrent Neural Network for Gesture Recognition on Embedded GPUs

TL;DR

The paper tackles gesture recognition on energy-constrained edge devices by deploying a spiking recurrent neural network with liquid time constant neurons on embedded GPUs. It leverages event-based DVS data, PyTorch-based training with forward propagation through time (FPTT), and a four-layer LTC-SRNN to balance temporal accuracy with power efficiency. Key contributions include a comprehensive cross-device benchmark showing Jetson Nano achieving strong energy efficiency relative to a high-end GPU and demonstrating that batch processing can boost frame rates without sacrificing accuracy, along with a detailed inference workflow for edge deployment. The work has practical significance for real-time human-computer interaction on portable devices, enabling robust temporal-spatial interpretation with low energy budgets.

Abstract

Implementing AI algorithms on event-based embedded devices enables real-time processing of data, minimizes latency, and enhances power efficiency in edge computing. This research explores the deployment of a spiking recurrent neural network (SRNN) with liquid time constant neurons for gesture recognition. We focus on the energy efficiency and computational efficacy of NVIDIA Jetson Nano embedded GPU platforms. The embedded GPU showcases a 14-fold increase in power efficiency relative to a conventional GPU, making a compelling argument for its use in energy-constrained applications. The study's empirical findings also highlight that batch processing significantly boosts frame rates across various batch sizes while maintaining accuracy levels well above the baseline. These insights validate the SRNN with liquid time constant neurons as a robust model for interpreting temporal-spatial data in gesture recognition, striking a critical balance between processing speed and power frugality.
Paper Structure (12 sections, 8 equations, 5 figures, 2 tables)

This paper contains 12 sections, 8 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. SNN on Embedded GPU
  3. Baseline network
  4. Advantage of LTC-based SRNNs
  5. Inference on AI edge device
  6. Inference process
  7. Experimental results
  8. Inference on standard GPU
  9. Inference on Jetson Nano
  10. Conclusion
  11. ACKNOWLEDGEMENTS
  12. Appendix A

Figures (5)

  • Figure 1: Three common types of spiking neuron models.
  • Figure 2: Inference procedure. a) preprocessing the raw data. b) Inference network including four spiking recurrent neural layers.
  • Figure 3: Experimental workflow.
  • Figure 4: Batch processing experimental results. Panels compare frame rate and accuracy for different batch sizes and $T$ values when using (a) the local GPU or (b) the embedded GPU.
  • Figure 5: Power consumption of the Jetson Nano. Batch size indicated by node width.