PromptGCN: Bridging Subgraph Gaps in Lightweight GCNs

TL;DR

Positive experimental results on seven largescale graphs demonstrate that PromptGCN exhibits superior performance compared to baselines, and it can be easily combined with any subgraph sampling method to obtain a lightweight GCN model with higher accuracy.

Abstract

Graph Convolutional Networks (GCNs) are widely used in graph-based applications, such as social networks and recommendation systems. Nevertheless, large-scale graphs or deep aggregation layers in full-batch GCNs consume significant GPU memory, causing out of memory (OOM) errors on mainstream GPUs (e.g., 29GB memory consumption on the Ogbnproducts graph with 5 layers). The subgraph sampling methods reduce memory consumption to achieve lightweight GCNs by partitioning the graph into multiple subgraphs and sequentially training GCNs on each subgraph. However, these methods yield gaps among subgraphs, i.e., GCNs can only be trained based on subgraphs instead of global graph information, which reduces the accuracy of GCNs. In this paper, we propose PromptGCN, a novel prompt-based lightweight GCN model to bridge the gaps among subgraphs. First, the learnable prompt embeddings are designed to obtain global information. Then, the prompts are attached into each subgraph to transfer the global information among subgraphs. Extensive experimental results on seven largescale graphs demonstrate that PromptGCN exhibits superior performance compared to baselines. Notably, PromptGCN improves the accuracy of subgraph sampling methods by up to 5.48% on the Flickr dataset. Overall, PromptGCN can be easily combined with any subgraph sampling method to obtain a lightweight GCN model with higher accuracy.

Figures (9)

• Figure 1: The GPU memory consumption of running full-batch GCN (blue) and ClusterGCN (green) in NVIDIA 3090 GPU on Obgn-products dataset. Full-batch GCN causes an out of memory (OOM) error with the number of layers and the hidden dimension increase. In contrast, the subgraph sampling method has lower memory consumption than full-batch GCN.
• Figure 2: Impact of the local receptive field of subgraphs on test accuracy (Ogbn-products and Reddit datasets).
• Figure 3: The overall process of PromptGCN includes: Graph Partitioning, Prompt Selection, Prompt Attachment, Prompt Sharing, and PromptGCN for Downstream Taks. First, in the Graph Partitioning module, the global graph is partitioned into several subgraphs. Second, in the Prompt Selection module, the node features within each subgraph adaptively select global prompt embeddings. Third, in the Prompt Attachment module, the selected global prompt embeddings are attached to the node features. Fourth, in the Prompt Sharing module, the processed node-prompt embedding pairs are input into a GCN model that is trained sequentially across subgraphs, sharing global graph information across subgraphs. Finally, PromptGCN is applied to various downstream tasks (e.g., Node Classification and Link Prediction) to guide the model training process.
• Figure 4: Performance comparison of promptGCN with baselines on different layers. PromptGCN achieves the highest ACC across different layers.
• Figure 5: Memory consumption of PromptGCN and compared models with different numbers of layers.