Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LHGstore: An In-Memory Learned Graph Storage for Fast Updates and Analytics

Pengpeng Qiao, Zhiwei Zhang, Xinzhou Wang, Zhetao Li, Xiaochun Cao, Yang Cao

Abstract

Various real-world applications rely on in-memory dynamic graphs that must efficiently handle frequent updates while supporting low-latency analytics on evolving structures. Achieving both objectives remains challenging due to the trade-off between update efficiency and traversal locality, particularly under highly skewed degree distributions. This motivates the design of graph indexing schemes optimized for in-memory graph management on modern multi-core CPUs. We present LHGstore, a degree-aware Learned Hierarchical Graph storage that, for the first time, integrates learned indexing into graph management. LHGstore designs a two-level hierarchy that decouples vertex and edge access and further organizes each vertex's edges using data structures adaptive to its degree. Lightweight arrays are used for low-degree vertices to maximize traversal locality, while learned indexes are applied to high-degree vertices to improve update throughput. Extensive experiments show that LHGstore achieves 5.9-28.2$\times$ higher throughput and significantly faster analytics than SOTA in-memory graph storage systems.

LHGstore: An In-Memory Learned Graph Storage for Fast Updates and Analytics

Abstract

Various real-world applications rely on in-memory dynamic graphs that must efficiently handle frequent updates while supporting low-latency analytics on evolving structures. Achieving both objectives remains challenging due to the trade-off between update efficiency and traversal locality, particularly under highly skewed degree distributions. This motivates the design of graph indexing schemes optimized for in-memory graph management on modern multi-core CPUs. We present LHGstore, a degree-aware Learned Hierarchical Graph storage that, for the first time, integrates learned indexing into graph management. LHGstore designs a two-level hierarchy that decouples vertex and edge access and further organizes each vertex's edges using data structures adaptive to its degree. Lightweight arrays are used for low-degree vertices to maximize traversal locality, while learned indexes are applied to high-degree vertices to improve update throughput. Extensive experiments show that LHGstore achieves 5.9-28.2 higher throughput and significantly faster analytics than SOTA in-memory graph storage systems.
Paper Structure (20 sections, 9 figures, 3 tables, 3 algorithms)

This paper contains 20 sections, 9 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Motivation and Baseline
  4. Motivation
  5. Memory Access Patterns
  6. Typical Graph Storage
  7. Our Baseline Design
  8. LHGstore Design
  9. Limitations of LGstore
  10. Overview of LHGstore
  11. Vertex Index
  12. Edge Index
  13. Operations
  14. Experiments
  15. Experimental Setup
  16. ...and 5 more sections

Figures (9)

  • Figure 1: A high-level architecture for graph indexing. A vertex index keeps track of where the neighbors (nbrs) for all vertices, and a list of edge indexes for each vertex’s edges.
  • Figure 2: A typical graph storage.
  • Figure 3: An example of using ML models to predict the position within a sorted array for a given key.
  • Figure 4: An example of a leaf node in LGstore.
  • Figure 5: The structure of LHGstore.
  • ...and 4 more figures

Theorems & Definitions (1)

  • Definition 1