Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PathMLP: Smooth Path Towards High-order Homophily

Jiajun Zhou, Chenxuan Xie, Shengbo Gong, Jiaxu Qian, Shanqing Yu, Qi Xuan, Xiaoniu Yang

TL;DR

A lightweight model based on multi-layer perceptrons (MLP), named PathMLP, is proposed, which can encode messages carried by paths via simple transformation and concatenation operations, and effectively learn node representations in heterophilous graphs through adaptive path aggregation.

Abstract

Real-world graphs exhibit increasing heterophily, where nodes no longer tend to be connected to nodes with the same label, challenging the homophily assumption of classical graph neural networks (GNNs) and impeding their performance. Intriguingly, from the observation of heterophilous data, we notice that certain high-order information exhibits higher homophily, which motivates us to involve high-order information in node representation learning. However, common practices in GNNs to acquire high-order information mainly through increasing model depth and altering message-passing mechanisms, which, albeit effective to a certain extent, suffer from three shortcomings: 1) over-smoothing due to excessive model depth and propagation times; 2) high-order information is not fully utilized; 3) low computational efficiency. In this regard, we design a similarity-based path sampling strategy to capture smooth paths containing high-order homophily. Then we propose a lightweight model based on multi-layer perceptrons (MLP), named PathMLP, which can encode messages carried by paths via simple transformation and concatenation operations, and effectively learn node representations in heterophilous graphs through adaptive path aggregation. Extensive experiments demonstrate that our method outperforms baselines on 16 out of 20 datasets, underlining its effectiveness and superiority in alleviating the heterophily problem. In addition, our method is immune to over-smoothing and has high computational efficiency. The source code will be available in https://github.com/Graph4Sec-Team/PathMLP.

PathMLP: Smooth Path Towards High-order Homophily

TL;DR

A lightweight model based on multi-layer perceptrons (MLP), named PathMLP, is proposed, which can encode messages carried by paths via simple transformation and concatenation operations, and effectively learn node representations in heterophilous graphs through adaptive path aggregation.

Abstract

Real-world graphs exhibit increasing heterophily, where nodes no longer tend to be connected to nodes with the same label, challenging the homophily assumption of classical graph neural networks (GNNs) and impeding their performance. Intriguingly, from the observation of heterophilous data, we notice that certain high-order information exhibits higher homophily, which motivates us to involve high-order information in node representation learning. However, common practices in GNNs to acquire high-order information mainly through increasing model depth and altering message-passing mechanisms, which, albeit effective to a certain extent, suffer from three shortcomings: 1) over-smoothing due to excessive model depth and propagation times; 2) high-order information is not fully utilized; 3) low computational efficiency. In this regard, we design a similarity-based path sampling strategy to capture smooth paths containing high-order homophily. Then we propose a lightweight model based on multi-layer perceptrons (MLP), named PathMLP, which can encode messages carried by paths via simple transformation and concatenation operations, and effectively learn node representations in heterophilous graphs through adaptive path aggregation. Extensive experiments demonstrate that our method outperforms baselines on 16 out of 20 datasets, underlining its effectiveness and superiority in alleviating the heterophily problem. In addition, our method is immune to over-smoothing and has high computational efficiency. The source code will be available in https://github.com/Graph4Sec-Team/PathMLP.
Paper Structure (31 sections, 1 theorem, 17 equations, 6 figures, 7 tables, 1 algorithm)

This paper contains 31 sections, 1 theorem, 17 equations, 6 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Notations
  5. Homophily
  6. Feature Similarity-based Path Sampling
  7. Path Sampling and Observation
  8. Theoretical Analysis of Path Sampling
  9. Methodology
  10. Model: PathMLP
  11. Variant: PathMLP+
  12. Experiments
  13. Dataset
  14. Baselines
  15. Experimental Settings
  16. ...and 16 more sections

Key Result

Theorem 1

The discrepancy between the smooth path sampling and the expected path selection will exhibit exponential decay as the number of sampled paths increases. In other words, as the number of samples increases, the smooth paths actually sampled will approach the expected path selection results with arbit

Figures (6)

  • Figure 1: Average edge homophily in different order. The computation of this metric is defined in Eq.(\ref{['eq: avg-edge-homo']}).
  • Figure 2: Statistics of the average order of candidate paths for each node obtained by different path sampling strategies.
  • Figure 3: Illustration of the PathMLP framework.
  • Figure 4: Impact of hyper-parameters in PathMLP+.
  • Figure 5: Average running time per epoch (ms).
  • ...and 1 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof