Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AGHINT: Attribute-Guided Representation Learning on Heterogeneous Information Networks with Transformer

Jinhui Yuan, Shan Lu, Peibo Duan, Jieyue He

TL;DR

This work addresses the poor performance of heterogeneous information network models when target node attributes diverge from neighbors. It proposes AGHINT, a Transformer-based framework featuring an attribute-guided transformer (AGT) and an attribute-guided message weighting (AGM) to incorporate attribute-similar neighbor information and reweight messages by attribute disparity, enabling effective long-range, attribute-aware aggregation beyond the original topology. Empirically, AGHINT outperforms state-of-the-art baselines on three real-world HIN benchmarks, with ablation and case studies showing the mechanisms’ contributions and their heightened impact as attribute disparity increases. The study demonstrates that explicit attribute guidance is crucial for robust heterogeneous graph learning and provides practical insights for building attribute-aware graph transformers.

Abstract

Recently, heterogeneous graph neural networks (HGNNs) have achieved impressive success in representation learning by capturing long-range dependencies and heterogeneity at the node level. However, few existing studies have delved into the utilization of node attributes in heterogeneous information networks (HINs). In this paper, we investigate the impact of inter-node attribute disparities on HGNNs performance within the benchmark task, i.e., node classification, and empirically find that typical models exhibit significant performance decline when classifying nodes whose attributes markedly differ from their neighbors. To alleviate this issue, we propose a novel Attribute-Guided heterogeneous Information Networks representation learning model with Transformer (AGHINT), which allows a more effective aggregation of neighbor node information under the guidance of attributes. Specifically, AGHINT transcends the constraints of the original graph structure by directly integrating higher-order similar neighbor features into the learning process and modifies the message-passing mechanism between nodes based on their attribute disparities. Extensive experimental results on three real-world heterogeneous graph benchmarks with target node attributes demonstrate that AGHINT outperforms the state-of-the-art.

AGHINT: Attribute-Guided Representation Learning on Heterogeneous Information Networks with Transformer

TL;DR

This work addresses the poor performance of heterogeneous information network models when target node attributes diverge from neighbors. It proposes AGHINT, a Transformer-based framework featuring an attribute-guided transformer (AGT) and an attribute-guided message weighting (AGM) to incorporate attribute-similar neighbor information and reweight messages by attribute disparity, enabling effective long-range, attribute-aware aggregation beyond the original topology. Empirically, AGHINT outperforms state-of-the-art baselines on three real-world HIN benchmarks, with ablation and case studies showing the mechanisms’ contributions and their heightened impact as attribute disparity increases. The study demonstrates that explicit attribute guidance is crucial for robust heterogeneous graph learning and provides practical insights for building attribute-aware graph transformers.

Abstract

Recently, heterogeneous graph neural networks (HGNNs) have achieved impressive success in representation learning by capturing long-range dependencies and heterogeneity at the node level. However, few existing studies have delved into the utilization of node attributes in heterogeneous information networks (HINs). In this paper, we investigate the impact of inter-node attribute disparities on HGNNs performance within the benchmark task, i.e., node classification, and empirically find that typical models exhibit significant performance decline when classifying nodes whose attributes markedly differ from their neighbors. To alleviate this issue, we propose a novel Attribute-Guided heterogeneous Information Networks representation learning model with Transformer (AGHINT), which allows a more effective aggregation of neighbor node information under the guidance of attributes. Specifically, AGHINT transcends the constraints of the original graph structure by directly integrating higher-order similar neighbor features into the learning process and modifies the message-passing mechanism between nodes based on their attribute disparities. Extensive experimental results on three real-world heterogeneous graph benchmarks with target node attributes demonstrate that AGHINT outperforms the state-of-the-art.
Paper Structure (24 sections, 16 equations, 5 figures, 2 tables)

This paper contains 24 sections, 16 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Heterogeneous Graphs Neural Networks
  4. Graph Transformers
  5. Preliminary
  6. Heterogeneous Information Network
  7. Graph Neural Networks
  8. Motivation
  9. Methodology
  10. Node Attribute Disparities Ascertainment
  11. Attribute-Guided Message-Weighting Module
  12. Attribute-Guided Similar Node Completion Module
  13. Training Objective
  14. Experiment
  15. Experimental Setups
  16. ...and 9 more sections

Figures (5)

  • Figure 1: Motivation verification on movie and citation datasets. The variation in classification accuracy for target nodes with different neighborhood attribute disparities indicates typical message-passing GNNs show a significant performance decline in classifying target nodes that exhibit significant attribute disparities with their neighboring nodes.
  • Figure 2: The overall architecture of AGHINT.
  • Figure 3: Results of different model variants.
  • Figure 4: Results of different hyperparameter settings.
  • Figure 5: Performance improvement by AGHINT on DBLP with varying neighborhood attribute disparities.