Ripple: Scalable Incremental GNN Inferencing on Large Streaming Graphs

TL;DR

Ripple tackles the challenge of real-time GNN inference on large streaming graphs by introducing an exact incremental propagation model that applies deltas to embeddings along a look-forward propagation tree. It leverages per-hop mailboxes and delta messages to update only the affected portions of the graph, significantly reducing recomputation compared to layer-wise recomputation. The framework supports both single-machine and distributed deployments, achieving orders of magnitude improvements in throughput (up to ~28k updates/sec on sparse graphs and ~30x gains in distributed setups) and low latencies suitable for near-real-time applications. By focusing on linear aggregation functions and providing a robust comparison against SOTA baselines, Ripple demonstrates practical viability for large-scale dynamic graphs in domains like finance, traffic, and social networks.

Abstract

Most real-world graphs are dynamic in nature, with continuous and rapid updates to the graph topology, and vertex and edge properties. Such frequent updates pose significant challenges for inferencing over Graph Neural Networks (GNNs). Current approaches that perform vertex-wise and layer-wise inferencing are impractical for dynamic graphs as they cause redundant computations, expand to large neighborhoods, and incur high communication costs for distributed setups, resulting in slow update propagation that often exceeds real-time latency requirements. This motivates the need for streaming GNN inference frameworks that are efficient and accurate over large, dynamic graphs. We propose Ripple, a framework that performs fast incremental updates of embeddings arising due to updates to the graph topology or vertex features. Ripple provides a generalized incremental programming model, leveraging the properties of the underlying aggregation functions employed by GNNs to efficiently propagate updates to the affected neighborhood and compute the exact new embeddings. Besides a single-machine design, we also extend this execution model to distributed inferencing, to support large graphs that do not fit in a single machine's memory. Ripple on a single machine achieves up to $\approx28000$ updates/sec for sparse graphs like Arxiv and $\approx1200$ updates/sec for larger and denser graphs like Products, with latencies of $0.1$ms--$1$s that are required for near-realtime applications. The distributed version of Ripple offers up to $\approx30\times$ better throughput over the baselines, due to $70\times$ lower communication costs during updates.

Figures (13)

• Figure 1: Vertex-wise vs. Layer-wise inferencing on static graphs.
• Figure 2: Performance and accuracy trade-offs for vertex-wise and layer-wise inferencing using $3$-Layer SAGEConv model.
• Figure 3: Cascading effect on the vertex embeddings due to an edge addition during GNN inference.
• Figure 4: Contrasting (b) Recomputation (RC) and (c)Ripple's Incremental Computation for (a) Edge addition in graph. Dark orange indicates vertex embeddings being updated in a hop.
• Figure 5: Using messages in the mailbox to update the hop-2 embeddings of vertex $D$ (in orange). The effect of the old embedding of $A$ is negated and replaced with the new embedding.