Hybrid variable spiking graph neural networks for energy-efficient scientific machine learning

TL;DR

This work tackles the energy demands of graph neural networks applied to mechanical-structure data by introducing Hybrid Variable Spiking Graph Neural Networks (HVS-GNNs) that embed Variable Spiking Neurons (VSNs) to enable sparse, event-driven computation. Training leverages surrogate backpropagation and a Spiking Loss Function to achieve a balance between prediction accuracy and reduced spiking activity, with VSN dynamics governed by memory, input, leakage, and a threshold. Across three regression tasks on microstructures and materials, HVS-GNNs either match or surpass conventional GNNs while exhibiting markedly lower spiking activity, particularly in the HVS-GNN2 variant, and SLF further reduces spikes with minimal loss in accuracy. This approach holds promise for energy-efficient GNNs on edge or neuromorphic hardware, expanding the practical deployment of physics-informed graph learning in engineering applications.

Abstract

Graph-based representations for samples of computational mechanics-related datasets can prove instrumental when dealing with problems like irregular domains or molecular structures of materials, etc. To effectively analyze and process such datasets, deep learning offers Graph Neural Networks (GNNs) that utilize techniques like message-passing within their architecture. The issue, however, is that as the individual graph scales and/ or GNN architecture becomes increasingly complex, the increased energy budget of the overall deep learning model makes it unsustainable and restricts its applications in applications like edge computing. To overcome this, we propose in this paper Hybrid Variable Spiking Graph Neural Networks (HVS-GNNs) that utilize Variable Spiking Neurons (VSNs) within their architecture to promote sparse communication and hence reduce the overall energy budget. VSNs, while promoting sparse event-driven computations, also perform well for regression tasks, which are often encountered in computational mechanics applications and are the main target of this paper. Three examples dealing with prediction of mechanical properties of material based on microscale/ mesoscale structures are shown to test the performance of the proposed HVS-GNNs in regression tasks. We have also compared the performance of HVS-GNN architectures with the performance of vanilla GNNs and GNNs utilizing leaky integrate and fire neurons. The results produced show that HVS-GNNs perform well for regression tasks, all while promoting sparse communication and, hence, energy efficiency.

Figures (14)

• Figure 1: Schematic to draw a parallel between spiking activity and computations involved (in synaptic operations) in deep learning models.
• Figure 2: Schematic showing the quality of node features in different variants of GNNs.
• Figure 3: Schematic for the base deep learning architecture used in Example-1.
• Figure 4: Plots for predicted vs true stiffness $k$ for both prediction evaluation cases in the first example.
• Figure 5: Plots for predicted vs true yield strength $f_y$ for both prediction evaluation cases in the first example.