LangTopo: Aligning Language Descriptions of Graphs with Tokenized Topological Modeling

TL;DR

A new framework, LangTopo, is introduced, which aligns graph structure modeling with natural language understanding at the token level and quantifies the graph structure modeling capabilities of GNNs and LLMs by constructing a codebook for the graph modality and performs consistency maximization.

Abstract

Recently, large language models (LLMs) have been widely researched in the field of graph machine learning due to their outstanding abilities in language comprehension and learning. However, the significant gap between natural language tasks and topological structure modeling poses a nonnegligible challenge. Specifically, since natural language descriptions are not sufficient for LLMs to understand and process graph-structured data, fine-tuned LLMs perform even worse than some traditional GNN models on graph tasks, lacking inherent modeling capabilities for graph structures. Existing research overly emphasizes LLMs' understanding of semantic information captured by external models, while inadequately exploring graph topological structure modeling, thereby overlooking the genuine capabilities that LLMs lack. Consequently, in this paper, we introduce a new framework, LangTopo, which aligns graph structure modeling with natural language understanding at the token level. LangTopo quantifies the graph structure modeling capabilities of GNNs and LLMs by constructing a codebook for the graph modality and performs consistency maximization. This process aligns the text description of LLM with the topological modeling of GNN, allowing LLM to learn the ability of GNN to capture graph structures, enabling LLM to handle graph-structured data independently. We demonstrate the effectiveness of our proposed method on multiple datasets.

Figures (4)

• Figure 1: Prompt: LLMs make predictions based solely on natural language descriptions. Access external: LLMs leverage external models (typically GNNs) to extract information for enhanced predictions. Ours LangTopo: Aligning the textual descriptive power of LLMs with the topological modeling capabilities of GNNs in terms of model processing and operation.
• Figure 2: The model architecture of our proposed LangTopo framework for graph structure learning.
• Figure 3: The distribution of codebook embeddings with different strategies on the unit hypersphere.
• Figure 4: Our investigation into diverse loss functions within the LangTopy architecture has substantiated the importance and efficacy of individual loss functions. The left figure examines the efficacy of node reconstruction and edge reconstruction loss functions, while the right figure delves into the importance of relaxed distributions and quantized embeddings in the learning process of LLMs.