LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework

TL;DR

This paper introduces LOGIN, a paradigm that integrates Large Language Models into GNN training via LLMs-as-Consultants. It selects uncertain nodes using MC dropout, constructs concise prompts containing semantic and topological cues, and uses LLM responses through a fault-tolerant coping mechanism that semantically updates correct predictions and topologically refines incorrect ones. Empirical results across six datasets show that even simple GNNs with LOGIN can rival advanced GNN architectures on both homophilic and heterophilic graphs, and that more powerful LLMs further boost performance. The work demonstrates the practicality of interactive LLM guidance to enhance graph learning while maintaining efficiency, and points to scalable strategies for broader node consultation and integration with stronger GNNs.

Abstract

Recent prevailing works on graph machine learning typically follow a similar methodology that involves designing advanced variants of graph neural networks (GNNs) to maintain the superior performance of GNNs on different graphs. In this paper, we aim to streamline the GNN design process and leverage the advantages of Large Language Models (LLMs) to improve the performance of GNNs on downstream tasks. We formulate a new paradigm, coined "LLMs-as-Consultants," which integrates LLMs with GNNs in an interactive manner. A framework named LOGIN (LLM Consulted GNN training) is instantiated, empowering the interactive utilization of LLMs within the GNN training process. First, we attentively craft concise prompts for spotted nodes, carrying comprehensive semantic and topological information, and serving as input to LLMs. Second, we refine GNNs by devising a complementary coping mechanism that utilizes the responses from LLMs, depending on their correctness. We empirically evaluate the effectiveness of LOGIN on node classification tasks across both homophilic and heterophilic graphs. The results illustrate that even basic GNN architectures, when employed within the proposed LLMs-as-Consultants paradigm, can achieve comparable performance to advanced GNNs with intricate designs. Our codes are available at https://github.com/QiaoYRan/LOGIN.

Figures (5)

• Figure 1: The paradigms for integrating LLMs with graphs.
• Figure 2: The pipeline of L[0.8]OG[0.8]IN.
• Figure 3: Ablation studies of L[0.8]OG[0.8]IN on PubMed and Texas.
• Figure 4: Case study of node 356 in Cora. Node 356 was first misclassified by GNN and selected as an uncertain node. The LLM consultation classified node 356 right along with a reasonable explanation. Then, the node representation was updated with the explanation. Finally, the retrained GNN managed to classify node 356 correctly.
• Figure 5: Case study of node 62 in Wisconsin. Node 62 was first misclassified by GNN and selected as an uncertain node. The LLM consultation classified node 62 wrong along with a self-contradictory explanation, which L[0.8]OG[0.8]IN left out. Then, edge pruning was performed around node 62. Finally, the retrained GNN managed to classify node 62 correctly.

Theorems & Definitions (2)

• definition 1: Graph-level Homophily Ratio
• definition 2: Node-level Homophily Ratio