Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ENADPool: The Edge-Node Attention-based Differentiable Pooling for Graph Neural Networks

Zhehan Zhao, Lu Bai, Lixin Cui, Ming Li, Yue Wang, Lixiang Xu, Edwin R. Hancock

TL;DR

The paper addresses the challenge of learning effective hierarchical representations in graph neural networks by introducing ENADPool, a hard-cluster pooling operation that simultaneously attends to node importance within clusters and edge importance between clusters. It pairs ENADPool with a Multi-distance GNN (MD-GNN) to mitigate over-smoothing by aggregating information from multiple hop distances. Key contributions include a hard node assignment mechanism, additive attention to reduce parameters, and an edge-attention scheme for weighted inter-cluster connectivity. Empirical results on standard graph-classification benchmarks show that ENADPool with MD-GNN outperforms existing pooling methods, with ablations confirming the importance of hard clustering, node/edge attention, and multi-distance aggregation. The approach offers a scalable, information-rich pooling paradigm that improves graph representations and classification performance in diverse datasets.

Abstract

Graph Neural Networks (GNNs) are powerful tools for graph classification. One important operation for GNNs is the downsampling or pooling that can learn effective embeddings from the node representations. In this paper, we propose a new hierarchical pooling operation, namely the Edge-Node Attention-based Differentiable Pooling (ENADPool), for GNNs to learn effective graph representations. Unlike the classical hierarchical pooling operation that is based on the unclear node assignment and simply computes the averaged feature over the nodes of each cluster, the proposed ENADPool not only employs a hard clustering strategy to assign each node into an unique cluster, but also compress the node features as well as their edge connectivity strengths into the resulting hierarchical structure based on the attention mechanism after each pooling step. As a result, the proposed ENADPool simultaneously identifies the importance of different nodes within each separated cluster and edges between corresponding clusters, that significantly addresses the shortcomings of the uniform edge-node based structure information aggregation arising in the classical hierarchical pooling operation. Moreover, to mitigate the over-smoothing problem arising in existing GNNs, we propose a Multi-distance GNN (MD-GNN) model associated with the proposed ENADPool operation, allowing the nodes to actively and directly receive the feature information from neighbors at different random walk steps. Experiments demonstrate the effectiveness of the MD-GNN associated with the proposed ENADPool.

ENADPool: The Edge-Node Attention-based Differentiable Pooling for Graph Neural Networks

TL;DR

The paper addresses the challenge of learning effective hierarchical representations in graph neural networks by introducing ENADPool, a hard-cluster pooling operation that simultaneously attends to node importance within clusters and edge importance between clusters. It pairs ENADPool with a Multi-distance GNN (MD-GNN) to mitigate over-smoothing by aggregating information from multiple hop distances. Key contributions include a hard node assignment mechanism, additive attention to reduce parameters, and an edge-attention scheme for weighted inter-cluster connectivity. Empirical results on standard graph-classification benchmarks show that ENADPool with MD-GNN outperforms existing pooling methods, with ablations confirming the importance of hard clustering, node/edge attention, and multi-distance aggregation. The approach offers a scalable, information-rich pooling paradigm that improves graph representations and classification performance in diverse datasets.

Abstract

Graph Neural Networks (GNNs) are powerful tools for graph classification. One important operation for GNNs is the downsampling or pooling that can learn effective embeddings from the node representations. In this paper, we propose a new hierarchical pooling operation, namely the Edge-Node Attention-based Differentiable Pooling (ENADPool), for GNNs to learn effective graph representations. Unlike the classical hierarchical pooling operation that is based on the unclear node assignment and simply computes the averaged feature over the nodes of each cluster, the proposed ENADPool not only employs a hard clustering strategy to assign each node into an unique cluster, but also compress the node features as well as their edge connectivity strengths into the resulting hierarchical structure based on the attention mechanism after each pooling step. As a result, the proposed ENADPool simultaneously identifies the importance of different nodes within each separated cluster and edges between corresponding clusters, that significantly addresses the shortcomings of the uniform edge-node based structure information aggregation arising in the classical hierarchical pooling operation. Moreover, to mitigate the over-smoothing problem arising in existing GNNs, we propose a Multi-distance GNN (MD-GNN) model associated with the proposed ENADPool operation, allowing the nodes to actively and directly receive the feature information from neighbors at different random walk steps. Experiments demonstrate the effectiveness of the MD-GNN associated with the proposed ENADPool.
Paper Structure (13 sections, 20 equations, 4 figures, 5 tables)

This paper contains 13 sections, 20 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. The Literature Review of Related works
  3. The Graph Convolutional Neural Networks
  4. The Graph Pooling Operation
  5. The Graph Attention Mechanism
  6. The Proposed Methods
  7. The Proposed ENADPool Operation
  8. The MD-GNN Model associated with the Proposed ENADPool
  9. Experiments
  10. Baselines and Experimental Settings
  11. Results and Discussions
  12. Ablation Experiments
  13. Conclusions

Figures (4)

  • Figure 1: The architecture of the MD-GNN model associated with the proposed ENADPool.
  • Figure 2: The process of aggregating the nodes for the coarsened graph through node attentions.
  • Figure 3: The process of aggregating the edges for the coarsened graph through edge attentions.
  • Figure 4: Left: The architecture of the MD-GNN. Right: An illustration of the MD convolution process, the different colors denote the nodes that participate the convolution operation with node $0$ and can arrive in node $0$ with different steps, e.g., the 1, 2 and 3 steps for this instance.