Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spiking Graph Neural Network on Riemannian Manifolds

Li Sun, Zhenhao Huang, Qiqi Wan, Hao Peng, Philip S. Yu

TL;DR

This work designs a new spiking neuron on geodesically complete manifolds with the diffeomorphism, so that BPTT regarding the spikes is replaced by the proposed differentiation via manifold, and shows that MSG approximates a solver of the manifold ordinary differential equation.

Abstract

Graph neural networks (GNNs) have become the dominant solution for learning on graphs, the typical non-Euclidean structures. Conventional GNNs, constructed with the Artificial Neuron Network (ANN), have achieved impressive performance at the cost of high computation and energy consumption. In parallel, spiking GNNs with brain-like spiking neurons are drawing increasing research attention owing to the energy efficiency. So far, existing spiking GNNs consider graphs in Euclidean space, ignoring the structural geometry, and suffer from the high latency issue due to Back-Propagation-Through-Time (BPTT) with the surrogate gradient. In light of the aforementioned issues, we are devoted to exploring spiking GNN on Riemannian manifolds, and present a Manifold-valued Spiking GNN (MSG). In particular, we design a new spiking neuron on geodesically complete manifolds with the diffeomorphism, so that BPTT regarding the spikes is replaced by the proposed differentiation via manifold. Theoretically, we show that MSG approximates a solver of the manifold ordinary differential equation. Extensive experiments on common graphs show the proposed MSG achieves superior performance to previous spiking GNNs and energy efficiency to conventional GNNs.

Spiking Graph Neural Network on Riemannian Manifolds

TL;DR

This work designs a new spiking neuron on geodesically complete manifolds with the diffeomorphism, so that BPTT regarding the spikes is replaced by the proposed differentiation via manifold, and shows that MSG approximates a solver of the manifold ordinary differential equation.

Abstract

Graph neural networks (GNNs) have become the dominant solution for learning on graphs, the typical non-Euclidean structures. Conventional GNNs, constructed with the Artificial Neuron Network (ANN), have achieved impressive performance at the cost of high computation and energy consumption. In parallel, spiking GNNs with brain-like spiking neurons are drawing increasing research attention owing to the energy efficiency. So far, existing spiking GNNs consider graphs in Euclidean space, ignoring the structural geometry, and suffer from the high latency issue due to Back-Propagation-Through-Time (BPTT) with the surrogate gradient. In light of the aforementioned issues, we are devoted to exploring spiking GNN on Riemannian manifolds, and present a Manifold-valued Spiking GNN (MSG). In particular, we design a new spiking neuron on geodesically complete manifolds with the diffeomorphism, so that BPTT regarding the spikes is replaced by the proposed differentiation via manifold. Theoretically, we show that MSG approximates a solver of the manifold ordinary differential equation. Extensive experiments on common graphs show the proposed MSG achieves superior performance to previous spiking GNNs and energy efficiency to conventional GNNs.

Paper Structure

This paper contains 70 sections, 5 theorems, 48 equations, 8 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Present work.
  3. Contributions.
  4. Related Work
  5. ANN-based GNNs (Euclidean and Riemannian).
  6. Spiking Neural Networks (SNNs) & Spiking GNNs.
  7. Preliminaries
  8. Riemannian Geometry & Riemannian Manifold.
  9. Graph & Riemannian Graph Representation Learning.
  10. Spiking Neural Network.
  11. Methodology: Manifold-valued Spiking GNN
  12. Manifold Spiking Layer
  13. Unified Formulation.
  14. Incorporating structural information.
  15. Diffeomorphism between manifolds.
  16. ...and 55 more sections

Key Result

Theorem 4.1

Let $\mathcal{L}$ be the scalar-valued function, and $\mathbf{z}^l$ is the output of $l$-th layer with parameter $\mathbf{W}^l$, which is delivered by tangent vector $\mathbf{v}^l$. Then, the gradient of function $\mathcal{L}$ w.r.t $\mathbf{W}^l$ is given as follows: where $\phi^{l-1}(\cdot)=\operatorname{Exp}_{\mathbf{z}^{l-1}}(\cdot)$, $\psi^{l}(\cdot)=\operatorname{Exp}_{(\cdot)}({\mathbf{v}^

Figures (8)

  • Figure 1: MSG conducts parallel forwarding and enables a new training algorithm alleviating the high latency issue.
  • Figure 2: Manifold Spiking Layer. It conducts parallel forwarding of spike trains and manifold representations, and creates an alternative backward pass (red dashed line). The backward gradient with $\frac{\partial \mathbf{v}^{l-1}}{\partial \mathbf{W}^l}$, $D_{\mathbf{v}^{l-1}}\phi^{l-1}$ and $\nabla_{\mathbf{z}^l} \mathcal{L}$ will be introduced in Sec. \ref{['sec:backward']}.
  • Figure 3: Charts given by the logarithmic map.
  • Figure 4: Backward time and gradient norm for node classification on Computer.
  • Figure 5: Visualization on $\mathbb S^1 \times \mathbb S^1$
  • ...and 3 more figures

Theorems & Definitions (9)

  • Theorem 4.1: Backward Gradient
  • Definition 5.1: Dynamic Chart Solver lou2020neural
  • Theorem 5.2: MSG as Dynamic Chart Solver
  • proof
  • Lemma B.1: Differential 1-form of a smooth function
  • Lemma B.2: Communication
  • Lemma B.3: Pullback of a sum and a product
  • proof
  • proof