Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spiking Heterogeneous Graph Attention Networks

Buqing Cao, Qian Peng, Xiang Xie, Liang Chen, Min Shi, Jianxun Liu

TL;DR

SpikingHAN addresses the high computational cost of heterogeneous graph neural networks by marrying brain-inspired spiking neural networks with metapath-based heterogeneous graph attention. It uses a single-layer shared-parameter graph convolution to aggregate metapath-neighborhoods, followed by semantic-level attention to fuse meta-path semantics, and then encodes the heterogeneous information into a binarized spike sequence through a spike-based neural network for prediction. The approach yields competitive node classification performance while reducing model parameters, memory footprint, inference time, and energy consumption on three real-world datasets. This work demonstrates a practical, energy-efficient pathway for deploying HGNNs on resource-constrained devices and paves the way for further exploration of SNNs in heterogeneous graph learning.

Abstract

Real-world graphs or networks are usually heterogeneous, involving multiple types of nodes and relationships. Heterogeneous graph neural networks (HGNNs) can effectively handle these diverse nodes and edges, capturing heterogeneous information within the graph, thus exhibiting outstanding performance. However, most methods of HGNNs usually involve complex structural designs, leading to problems such as high memory usage, long inference time, and extensive consumption of computing resources. These limitations pose certain challenges for the practical application of HGNNs, especially for resource-constrained devices. To mitigate this issue, we propose the Spiking Heterogeneous Graph Attention Networks (SpikingHAN), which incorporates the brain-inspired and energy-saving properties of Spiking Neural Networks (SNNs) into heterogeneous graph learning to reduce the computing cost without compromising the performance. Specifically, SpikingHAN aggregates metapath-based neighbor information using a single-layer graph convolution with shared parameters. It then employs a semantic-level attention mechanism to capture the importance of different meta-paths and performs semantic aggregation. Finally, it encodes the heterogeneous information into a spike sequence through SNNs, simulating bioinformatic processing to derive a binarized 1-bit representation of the heterogeneous graph. Comprehensive experimental results from three real-world heterogeneous graph datasets show that SpikingHAN delivers competitive node classification performance. It achieves this with fewer parameters, quicker inference, reduced memory usage, and lower energy consumption. Code is available at https://github.com/QianPeng369/SpikingHAN.

Spiking Heterogeneous Graph Attention Networks

TL;DR

SpikingHAN addresses the high computational cost of heterogeneous graph neural networks by marrying brain-inspired spiking neural networks with metapath-based heterogeneous graph attention. It uses a single-layer shared-parameter graph convolution to aggregate metapath-neighborhoods, followed by semantic-level attention to fuse meta-path semantics, and then encodes the heterogeneous information into a binarized spike sequence through a spike-based neural network for prediction. The approach yields competitive node classification performance while reducing model parameters, memory footprint, inference time, and energy consumption on three real-world datasets. This work demonstrates a practical, energy-efficient pathway for deploying HGNNs on resource-constrained devices and paves the way for further exploration of SNNs in heterogeneous graph learning.

Abstract

Real-world graphs or networks are usually heterogeneous, involving multiple types of nodes and relationships. Heterogeneous graph neural networks (HGNNs) can effectively handle these diverse nodes and edges, capturing heterogeneous information within the graph, thus exhibiting outstanding performance. However, most methods of HGNNs usually involve complex structural designs, leading to problems such as high memory usage, long inference time, and extensive consumption of computing resources. These limitations pose certain challenges for the practical application of HGNNs, especially for resource-constrained devices. To mitigate this issue, we propose the Spiking Heterogeneous Graph Attention Networks (SpikingHAN), which incorporates the brain-inspired and energy-saving properties of Spiking Neural Networks (SNNs) into heterogeneous graph learning to reduce the computing cost without compromising the performance. Specifically, SpikingHAN aggregates metapath-based neighbor information using a single-layer graph convolution with shared parameters. It then employs a semantic-level attention mechanism to capture the importance of different meta-paths and performs semantic aggregation. Finally, it encodes the heterogeneous information into a spike sequence through SNNs, simulating bioinformatic processing to derive a binarized 1-bit representation of the heterogeneous graph. Comprehensive experimental results from three real-world heterogeneous graph datasets show that SpikingHAN delivers competitive node classification performance. It achieves this with fewer parameters, quicker inference, reduced memory usage, and lower energy consumption. Code is available at https://github.com/QianPeng369/SpikingHAN.
Paper Structure (24 sections, 14 equations, 6 figures, 3 tables, 1 algorithm)

This paper contains 24 sections, 14 equations, 6 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Heterogeneous Graph Neural Network
  4. Spiking Neural Networks
  5. Preliminary
  6. Definition 1: Heterogeneous Graph.
  7. Definition 2: Metapath-based Neighbor.
  8. Definition 3: Spiking Neural Network.
  9. Methodology
  10. Shared Convolution and Attention
  11. Fully Connected Layer and SNN
  12. Spikes Statistic and Prediction
  13. Experiment
  14. Experimental Settings
  15. Dataset description.
  16. ...and 9 more sections

Figures (6)

  • Figure 1: Diagram of a heterogeneous graph and comparison between our model and HAN
  • Figure 2: The overall framework of SpikingHAN
  • Figure 3: Model training time on different datasets
  • Figure 4: GPU energy consumption per epoch during model training
  • Figure 5: The impact of different spiking neurons
  • ...and 1 more figures