Heterophily-Aware Graph Attention Network

TL;DR

This work tackles the heterophily challenge in graph neural networks by introducing edge-type aware weighting and a local distribution explorer. The core method, HA-GAT, uses a heterophily preference matrix derived from edge types and a gradient-scaled parsing function to compute attention, while a two-layer explorer network derives a local category distribution that informs edge heterophily through a learned m_{ij}. The approach yields state-of-the-art results across eight datasets with varying homophily ratios in both supervised and semi-supervised node classification, and provides interpretable insights through LAP visualizations and ablations. The proposed combination of edge-level heterophily modeling and local-distribution-driven attention offers robust performance when homophily varies and computational efficiency is maintained, making it applicable to broad graph learning tasks.

Abstract

Graph Neural Networks (GNNs) have shown remarkable success in graph representation learning. Unfortunately, current weight assignment schemes in standard GNNs, such as the calculation based on node degrees or pair-wise representations, can hardly be effective in processing the networks with heterophily, in which the connected nodes usually possess different labels or features. Existing heterophilic GNNs tend to ignore the modeling of heterophily of each edge, which is also a vital part in tackling the heterophily problem. In this paper, we firstly propose a heterophily-aware attention scheme and reveal the benefits of modeling the edge heterophily, i.e., if a GNN assigns different weights to edges according to different heterophilic types, it can learn effective local attention patterns, which enable nodes to acquire appropriate information from distinct neighbors. Then, we propose a novel Heterophily-Aware Graph Attention Network (HA-GAT) by fully exploring and utilizing the local distribution as the underlying heterophily, to handle the networks with different homophily ratios. To demonstrate the effectiveness of the proposed HA-GAT, we analyze the proposed heterophily-aware attention scheme and local distribution exploration, by seeking for an interpretation from their mechanism. Extensive results demonstrate that our HA-GAT achieves state-of-the-art performances on eight datasets with different homophily ratios in both the supervised and semi-supervised node classification tasks.

Figures (9)

• Figure 1: Illustration of our Heterophily-Aware Graph Attention Network (HA-GAT).
• Figure 2: Visualization of the LAPs for a 2-layered HA-GAT(L) on Chameleon.
• Figure 3: Visualization of the LAPs for a 2-layered HA-GAT with different explorer networks on Actor.
• Figure 4: Results of 2-layered HA-GATs on the Chameleon dataset. (a) shows the accuracies of HA-GAT with different category dimension $t$. (b) is the heatmaps of the overall heterophily preference (Top) $\boldsymbol{M}=\sum_{e_{ij}}\mathbf{m_{ij}}$ and the overall node categories (Bottom) $N_T=\sum_{v_i}s_i$ with $t=3$. (c) and (d) are the LAPs of HA-GAT. (e) (g) are the heatmaps of HA-GAT with $t=5$.
• Figure 5: Layer-wise distributions of the edge weights learned by different models.