Bridging Local Details and Global Context in Text-Attributed Graphs

TL;DR

GraphBridge is proposed, a multi-granularity integration framework that bridges local and global perspectives by leveraging contextual textual information, enhancing fine-grained understanding of TAGs and a graph-aware token reduction module to tackle scalability and efficiency challenges.

Abstract

Representation learning on text-attributed graphs (TAGs) is vital for real-world applications, as they combine semantic textual and contextual structural information. Research in this field generally consist of two main perspectives: local-level encoding and global-level aggregating, respectively refer to textual node information unification (e.g., using Language Models) and structure-augmented modeling (e.g., using Graph Neural Networks). Most existing works focus on combining different information levels but overlook the interconnections, i.e., the contextual textual information among nodes, which provides semantic insights to bridge local and global levels. In this paper, we propose GraphBridge, a multi-granularity integration framework that bridges local and global perspectives by leveraging contextual textual information, enhancing fine-grained understanding of TAGs. Besides, to tackle scalability and efficiency challenges, we introduce a graphaware token reduction module. Extensive experiments across various models and datasets show that our method achieves state-of-theart performance, while our graph-aware token reduction module significantly enhances efficiency and solves scalability issues.

Figures (6)

• Figure 1: Illustration of the local-global integration in TAGs within a social network context. The words in pink emphasize the interconnection semantic relationship between them. (i) The local-level encoding module processes individual nodes' textual information into unified vectors; (ii) The global-level aggregating module enhances node features with structural information; (iii) Our method bridges these two perspectives through incorporating contextual textual information.
• Figure 2: Overview of the GraphBridge framework. Left: The Graph-Aware Token Reduction module which selectively retains crucial tokens, enhancing efficiency and scalability. Right: A detailed pipeline illustrates the integration process, where selected tokens undergo a cascaded structure that bridges local and global perspectives, leveraging contextual textual information to effectively refine node representations.
• Figure 3: Selecting the highest score token for each node in WikiCS dataset, with and without regularization. The x-axis means the highest importance score of token for each node, the y-axis indicates the number of nodes corresponding to each importance score.
• Figure 4: Sensitive analysis of the number of walk steps.
• Figure 5: Sensitive analysis of regularization term $\beta$.