Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoning

Mingqing Xiao, Yixin Zhu, Di He, Zhouchen Lin

TL;DR

This work tackles human-like graph reasoning within neural models by introducing GRSNN, a temporal-spiking neural network that uses synaptic delays to encode edge properties and the temporal domain as an additional dimension for processing paths. The model associates each graph node with a population of $n$ spiking neurons and learns delays $\boldsymbol{d}^q_r$ conditioned on relations and query type, enabling a neural generalization of path formulations akin to Dijkstra’s algorithm. Empirical results across transductive KG completion, inductive relation prediction, and homogeneous graph link prediction show competitive performance with significantly reduced parameter counts and promising energy efficiency (theoretically up to $\sim$20x savings) due to event-driven spiking. The work demonstrates interpretability of reasoning paths via gradient-based edge importance and beam search, and points to substantial practical impact for energy-efficient, neuromorphic graph reasoning on future hardware.

Abstract

Spiking neural networks (SNNs) are investigated as biologically inspired models of neural computation, distinguished by their computational capability and energy efficiency due to precise spiking times and sparse spikes with event-driven computation. A significant question is how SNNs can emulate human-like graph-based reasoning of concepts and relations, especially leveraging the temporal domain optimally. This paper reveals that SNNs, when amalgamated with synaptic delay and temporal coding, are proficient in executing (knowledge) graph reasoning. It is elucidated that spiking time can function as an additional dimension to encode relation properties via a neural-generalized path formulation. Empirical results highlight the efficacy of temporal delay in relation processing and showcase exemplary performance in diverse graph reasoning tasks. The spiking model is theoretically estimated to achieve $20\times$ energy savings compared to non-spiking counterparts, deepening insights into the capabilities and potential of biologically inspired SNNs for efficient reasoning. The code is available at https://github.com/pkuxmq/GRSNN.

Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoning

TL;DR

This work tackles human-like graph reasoning within neural models by introducing GRSNN, a temporal-spiking neural network that uses synaptic delays to encode edge properties and the temporal domain as an additional dimension for processing paths. The model associates each graph node with a population of spiking neurons and learns delays conditioned on relations and query type, enabling a neural generalization of path formulations akin to Dijkstra’s algorithm. Empirical results across transductive KG completion, inductive relation prediction, and homogeneous graph link prediction show competitive performance with significantly reduced parameter counts and promising energy efficiency (theoretically up to 20x savings) due to event-driven spiking. The work demonstrates interpretability of reasoning paths via gradient-based edge importance and beam search, and points to substantial practical impact for energy-efficient, neuromorphic graph reasoning on future hardware.

Abstract

Spiking neural networks (SNNs) are investigated as biologically inspired models of neural computation, distinguished by their computational capability and energy efficiency due to precise spiking times and sparse spikes with event-driven computation. A significant question is how SNNs can emulate human-like graph-based reasoning of concepts and relations, especially leveraging the temporal domain optimally. This paper reveals that SNNs, when amalgamated with synaptic delay and temporal coding, are proficient in executing (knowledge) graph reasoning. It is elucidated that spiking time can function as an additional dimension to encode relation properties via a neural-generalized path formulation. Empirical results highlight the efficacy of temporal delay in relation processing and showcase exemplary performance in diverse graph reasoning tasks. The spiking model is theoretically estimated to achieve energy savings compared to non-spiking counterparts, deepening insights into the capabilities and potential of biologically inspired SNNs for efficient reasoning. The code is available at https://github.com/pkuxmq/GRSNN.
Paper Structure (42 sections, 2 theorems, 18 equations, 8 figures, 12 tables)

This paper contains 42 sections, 2 theorems, 18 equations, 8 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Spiking Neural Networks
  4. Link Prediction of Graphs
  5. Synaptic Delay for Traditional Graph Algorithms
  6. Graph Reasoning Spiking Neural Network
  7. Model Overview
  8. GRSNN as Generalized Path Formulation
  9. Comparison with Graph Neural Networks
  10. Implementation Details
  11. Model Detail
  12. Link Prediction Detail
  13. Experiments
  14. Transductive Knowledge Graph Completion
  15. Analysis Results
  16. ...and 27 more sections

Key Result

Proposition 3.1

Katz Index, Personalized PageRank, and Graph Distance can be solved by GRSNN under specific settings.

Figures (8)

  • Figure 1: Depiction of spiking neural networks and knowledge graph reasoning. (a) A representation of biological neural circuits, showcasing spiking neurons, their inherent dynamics, synaptic interconnections, and the propagation of temporal spike trains. (b) The process of relational reasoning of concepts, exemplified through the link prediction task in knowledge graphs.
  • Figure 2: Schematic of GRSNN. (a) Illustration of the graph link prediction task. In GRSNN, (b) each graph entity node is associated with a cluster of spiking neurons, and (c) each relational edge corresponds to the synaptic connections between spiking neurons, with synaptic weight and delay. The weight can exhibit positive or negative values. The delay is contingent on the edge relation and query relation, representing the edge's property and the neuromodulation from the task goal. (d) Visualization of GRSNN. To predict a link, a constant current, dependent on the query relation, is injected into the spiking neurons of the source node, initiating the propagation of spike trains. After a specific time interval, the spike trains emanating from the target node are decoded to predict the query relation. (e) Depiction of the temporal domain serving as an additional dimension to process the properties of edges and paths in a network with more propagation paths. In the demonstrated network under a simplified setting where each input spike triggers an output spike for neurons, a spike from the source neuron will lead to four spikes from the target neuron, whose time varies corresponding to four propagation paths with different integrated properties of edges represented by synaptic delay.
  • Figure 3: Results of Transductive Knowledge Graph Completion on FB15k-237 and WN18RR. Lower values are preferable for MR, while higher values are desirable for MRR, HITS@1, and HITS@10. Detailed values can be found in \ref{['supp:sec:detailed_values']}.
  • Figure 4: Analytical Results for GRSNN. (a) Log-scale comparison of the parameter quantities across different methods, demonstrating the enhanced parameter efficiency of GRSNN. (b) Theoretical estimations of the number of ADD and MUL operations (log scale) and energy consumption on FB15k-237. GRSNN can achieve approximately $20\times$ energy reduction compared to its non-spiking counterpart.
  • Figure 5: Results of Inductive Relation Prediction on FB15k-237 and WN18RR. v1-v4 correspond to the four standard versions of inductive splits. Detailed values can be found in \ref{['supp:sec:detailed_values']}.
  • ...and 3 more figures

Theorems & Definitions (4)

  • Proposition 3.1
  • Proposition 2.1
  • proof
  • Remark 2.2