Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enabling Accelerators for Graph Computing

Kaustubh Shivdikar

TL;DR

This thesis addresses the challenges posed by SpGEMM by implementing a highly optimized hashing-based SpGEMM kernel tailored for a custom accelerator and designs state-of-the-art hardware accelerators capable of efficiently handling various GNN workloads.

Abstract

The advent of Graph Neural Networks (GNNs) has revolutionized the field of machine learning, offering a novel paradigm for learning on graph-structured data. Unlike traditional neural networks, GNNs are capable of capturing complex relationships and dependencies inherent in graph data, making them particularly suited for a wide range of applications including social network analysis, molecular chemistry, and network security. GNNs, with their unique structure and operation, present new computational challenges compared to conventional neural networks. This requires comprehensive benchmarking and a thorough characterization of GNNs to obtain insight into their computational requirements and to identify potential performance bottlenecks. In this thesis, we aim to develop a better understanding of how GNNs interact with the underlying hardware and will leverage this knowledge as we design specialized accelerators and develop new optimizations, leading to more efficient and faster GNN computations. A pivotal component within GNNs is the Sparse General Matrix-Matrix Multiplication (SpGEMM) kernel, known for its computational intensity and irregular memory access patterns. In this thesis, we address the challenges posed by SpGEMM by implementing a highly optimized hashing-based SpGEMM kernel tailored for a custom accelerator. Synthesizing these insights and optimizations, we design state-of-the-art hardware accelerators capable of efficiently handling various GNN workloads. Our accelerator architectures are built on our characterization of GNN computational demands, providing clear motivation for our approaches. This exploration into novel models underlines our comprehensive approach, as we strive to enable accelerators that are not just performant, but also versatile, able to adapt to the evolving landscape of graph computing.

Enabling Accelerators for Graph Computing

TL;DR

This thesis addresses the challenges posed by SpGEMM by implementing a highly optimized hashing-based SpGEMM kernel tailored for a custom accelerator and designs state-of-the-art hardware accelerators capable of efficiently handling various GNN workloads.

Abstract

The advent of Graph Neural Networks (GNNs) has revolutionized the field of machine learning, offering a novel paradigm for learning on graph-structured data. Unlike traditional neural networks, GNNs are capable of capturing complex relationships and dependencies inherent in graph data, making them particularly suited for a wide range of applications including social network analysis, molecular chemistry, and network security. GNNs, with their unique structure and operation, present new computational challenges compared to conventional neural networks. This requires comprehensive benchmarking and a thorough characterization of GNNs to obtain insight into their computational requirements and to identify potential performance bottlenecks. In this thesis, we aim to develop a better understanding of how GNNs interact with the underlying hardware and will leverage this knowledge as we design specialized accelerators and develop new optimizations, leading to more efficient and faster GNN computations. A pivotal component within GNNs is the Sparse General Matrix-Matrix Multiplication (SpGEMM) kernel, known for its computational intensity and irregular memory access patterns. In this thesis, we address the challenges posed by SpGEMM by implementing a highly optimized hashing-based SpGEMM kernel tailored for a custom accelerator. Synthesizing these insights and optimizations, we design state-of-the-art hardware accelerators capable of efficiently handling various GNN workloads. Our accelerator architectures are built on our characterization of GNN computational demands, providing clear motivation for our approaches. This exploration into novel models underlines our comprehensive approach, as we strive to enable accelerators that are not just performant, but also versatile, able to adapt to the evolving landscape of graph computing.
Paper Structure (92 sections, 13 equations, 41 figures, 12 tables, 4 algorithms)

This paper contains 92 sections, 13 equations, 41 figures, 12 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Challenges in Accelerating Graph Computing
  4. Scalability Concerns in Graph Neural Networks
  5. Task-Level Parallelism in Graph Computations
  6. Data Spatial Locality and Computational Irregularity
  7. Objectives and Contributions
  8. Dissertation Organization
  9. Fundamentals of Graph Computing and Accelerator Architectures
  10. Graph Theory Basics
  11. Graph Properties
  12. Types of Graphs
  13. Graph Representation
  14. Graph Transformation
  15. Machine Learning on Graphs
  16. ...and 77 more sections

Figures (41)

  • Figure 1.1: Graph Computing Applications
  • Figure 2.2: An example of a social network graph and its corresponding adjacency matrix. Each node in the graph is associated with a feature vector that contains the node's attributes.
  • Figure 2.3: Analysis of Graph Neural Networks, demonstrating the propagation of node properties influenced by the graph's topology.
  • Figure 4.4: Execution breakdown, reported as the percent of total execution time, for individual operations across the different workloads of GNNMark.
  • Figure 4.5: Breakdown of fp32 vs. int32 instructions across the different workloads in GNNMark.
  • ...and 36 more figures