Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Aspect Term Extraction using Spiking Neural Network

Abhishek Kumar Mishra, Arya Somasundaram, Anup Das, Nagarajan Kandasamy

TL;DR

The paper addresses the energy inefficiency of deep learning approaches for Aspect Term Extraction (ATE) by introducing SpikeATE, a Spiking Neural Network (SNN) that leverages sparse, event-driven computation and temporal encoding. It introduces a ternary spiking neuron with values $-1$, $0$, $1$ and a convolutional spike-encoding pipeline to perform token-level ATE, trained with surrogate-gradient-based spatio-temporal backpropagation. Across SemEval datasets, SpikeATE achieves competitive accuracy with substantial energy reductions (up to 41x) compared to strong DNN baselines, demonstrating practical viability for sustainable NLP. The work suggests a path toward deploying energy-efficient, neuromorphic NLP systems on future hardware while maintaining task performance.

Abstract

Aspect Term Extraction (ATE) identifies aspect terms in review sentences, a key subtask of sentiment analysis. While most existing approaches use energy-intensive deep neural networks (DNNs) for ATE as sequence labeling, this paper proposes a more energy-efficient alternative using Spiking Neural Networks (SNNs). Using sparse activations and event-driven inferences, SNNs capture temporal dependencies between words, making them suitable for ATE. The proposed architecture, SpikeATE, employs ternary spiking neurons and direct spike training fine-tuned with pseudo-gradients. Evaluated on four benchmark SemEval datasets, SpikeATE achieves performance comparable to state-of-the-art DNNs with significantly lower energy consumption. This highlights the use of SNNs as a practical and sustainable choice for ATE tasks.

Efficient Aspect Term Extraction using Spiking Neural Network

TL;DR

The paper addresses the energy inefficiency of deep learning approaches for Aspect Term Extraction (ATE) by introducing SpikeATE, a Spiking Neural Network (SNN) that leverages sparse, event-driven computation and temporal encoding. It introduces a ternary spiking neuron with values , , and a convolutional spike-encoding pipeline to perform token-level ATE, trained with surrogate-gradient-based spatio-temporal backpropagation. Across SemEval datasets, SpikeATE achieves competitive accuracy with substantial energy reductions (up to 41x) compared to strong DNN baselines, demonstrating practical viability for sustainable NLP. The work suggests a path toward deploying energy-efficient, neuromorphic NLP systems on future hardware while maintaining task performance.

Abstract

Aspect Term Extraction (ATE) identifies aspect terms in review sentences, a key subtask of sentiment analysis. While most existing approaches use energy-intensive deep neural networks (DNNs) for ATE as sequence labeling, this paper proposes a more energy-efficient alternative using Spiking Neural Networks (SNNs). Using sparse activations and event-driven inferences, SNNs capture temporal dependencies between words, making them suitable for ATE. The proposed architecture, SpikeATE, employs ternary spiking neurons and direct spike training fine-tuned with pseudo-gradients. Evaluated on four benchmark SemEval datasets, SpikeATE achieves performance comparable to state-of-the-art DNNs with significantly lower energy consumption. This highlights the use of SNNs as a practical and sustainable choice for ATE tasks.
Paper Structure (17 sections, 12 equations, 5 figures, 6 tables, 2 algorithms)

This paper contains 17 sections, 12 equations, 5 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Token-Level Classification
  5. Spiking Neural Network Architecture
  6. Training Spiking Neural Networks
  7. Ternary Spiking Neuron
  8. Development of SpikeATE
  9. Convolutional Spike Encoding Layer
  10. Spiking-Based Convolutional Layer
  11. Training Procedure for Spiking Neural Network
  12. Performance Evaluation
  13. Results and Ablation Study
  14. Detailed Analysis
  15. Computational Efficiency
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Review sentences illustrate the ATE task where aspect terms are highlighted in purple.
  • Figure 2: The update, fire, and reset cycle for an LIF neuron.
  • Figure 3: The SpikeATE architecture consisting of a convolutional spike encoding layer, multiple spike-based convolutional layers, and the output layer.
  • Figure 4: Convolutional spike encoding operation for the first two time steps.
  • Figure 5: Visualization of spiking activity of the SNN when presented with tokens from Review1.