TANGNN: a Concise, Scalable and Effective Graph Neural Networks with Top-m Attention Mechanism for Graph Representation Learning

TL;DR

TANGNN introduces a dual-aggregation graph neural network that combines a Top-$m$ attention mechanism with traditional neighborhood aggregation to efficiently capture both local and long-range information. By integrating a Top-$m$ efficient algorithm and flexible variants, it achieves strong performance across node classification, link prediction, graph regression, and a novel citation sentiment task using the ArXivNet dataset. The framework demonstrates faster convergence, reduced memory usage, and robustness on large graphs, with TANGNN-LC often outperforming state-of-the-art baselines. The work also provides a new benchmark for citation sentiment prediction in graphs, highlighting practical applicability to large-scale graph data analysis.

Abstract

In the field of deep learning, Graph Neural Networks (GNNs) and Graph Transformer models, with their outstanding performance and flexible architectural designs, have become leading technologies for processing structured data, especially graph data. Traditional GNNs often face challenges in capturing information from distant vertices effectively. In contrast, Graph Transformer models are particularly adept at managing long-distance node relationships. Despite these advantages, Graph Transformer models still encounter issues with computational and storage efficiency when scaled to large graph datasets. To address these challenges, we propose an innovative Graph Neural Network (GNN) architecture that integrates a Top-m attention mechanism aggregation component and a neighborhood aggregation component, effectively enhancing the model's ability to aggregate relevant information from both local and extended neighborhoods at each layer. This method not only improves computational efficiency but also enriches the node features, facilitating a deeper analysis of complex graph structures. Additionally, to assess the effectiveness of our proposed model, we have applied it to citation sentiment prediction, a novel task previously unexplored in the GNN field. Accordingly, we constructed a dedicated citation network, ArXivNet. In this dataset, we specifically annotated the sentiment polarity of the citations (positive, neutral, negative) to enable in-depth sentiment analysis. Our approach has shown superior performance across a variety of tasks including vertex classification, link prediction, sentiment prediction, graph regression, and visualization. It outperforms existing methods in terms of effectiveness, as demonstrated by experimental results on multiple datasets.

Figures (10)

• Figure 1: TANGNN Architecture
• Figure 2: Illustration of the Top-$m$ Efficient Algorithm. An auxiliary vector $a$ is introduced into the model to calculate the cosine similarity between node vector representations and the auxiliary vector, determining the similarity between nodes. This method utilizes the transfer property of similarity, reducing the necessary computational complexity. The value of $m$ is set to 2, the two most similar nodes to each node are its two nearest neighbors. The boundary nodes ($g_4$ and $g_5$) have their two nearest nodes as their most similar nodes.
• Figure 3: TANGNN-LC architecture
• Figure 4: TANGNN-FLC architecture
• Figure 5: Accuracy values for vertex classification tasks on Cora and PubMed datasets vary with $K$ values

Theorems & Definitions (3)

• Definition 1
• Definition 2
• Definition 3