Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conditional Local Feature Encoding for Graph Neural Networks

Yongze Wang, Haimin Zhang, Qiang Wu, Min Xu

TL;DR

The paper tackles the problem of increasing neighborhood dominance in deep GNNs, which leads to oversmoothing and reduced node discriminability. It presents Conditional Local Feature Encoding (CLFE), a per-layer plugin that concatenates a node’s previous-layer output with its current hidden state and applies a learnable linear transform to form a CLFE, which is added back to the update to preserve node-specific information. Empirically, CLFE improves performance across seven benchmark datasets and four graph tasks (node/edge classification, graph classification, and regression) when integrated with five popular GNNs, and it enables deeper architectures with consistent gains. This approach offers a general, task-agnostic method to mitigate neighborhood dominance and enhance depth in graph neural networks, with practical impact across diverse graph domains.

Abstract

Graph neural networks (GNNs) have shown great success in learning from graph-based data. The key mechanism of current GNNs is message passing, where a node's feature is updated based on the information passing from its local neighbourhood. A limitation of this mechanism is that node features become increasingly dominated by the information aggregated from the neighbourhood as we use more rounds of message passing. Consequently, as the GNN layers become deeper, adjacent node features tends to be similar, making it more difficult for GNNs to distinguish adjacent nodes, thereby, limiting the performance of GNNs. In this paper, we propose conditional local feature encoding (CLFE) to help prevent the problem of node features being dominated by the information from local neighbourhood. The idea of our method is to extract the node hidden state embedding from message passing process and concatenate it with the nodes feature from previous stage, then we utilise linear transformation to form a CLFE based on the concatenated vector. The CLFE will form the layer output to better preserve node-specific information, thus help to improve the performance of GNN models. To verify the feasibility of our method, we conducted extensive experiments on seven benchmark datasets for four graph domain tasks: super-pixel graph classification, node classification, link prediction, and graph regression. The experimental results consistently demonstrate that our method improves model performance across a variety of baseline GNN models for all four tasks.

Conditional Local Feature Encoding for Graph Neural Networks

TL;DR

The paper tackles the problem of increasing neighborhood dominance in deep GNNs, which leads to oversmoothing and reduced node discriminability. It presents Conditional Local Feature Encoding (CLFE), a per-layer plugin that concatenates a node’s previous-layer output with its current hidden state and applies a learnable linear transform to form a CLFE, which is added back to the update to preserve node-specific information. Empirically, CLFE improves performance across seven benchmark datasets and four graph tasks (node/edge classification, graph classification, and regression) when integrated with five popular GNNs, and it enables deeper architectures with consistent gains. This approach offers a general, task-agnostic method to mitigate neighborhood dominance and enhance depth in graph neural networks, with practical impact across diverse graph domains.

Abstract

Graph neural networks (GNNs) have shown great success in learning from graph-based data. The key mechanism of current GNNs is message passing, where a node's feature is updated based on the information passing from its local neighbourhood. A limitation of this mechanism is that node features become increasingly dominated by the information aggregated from the neighbourhood as we use more rounds of message passing. Consequently, as the GNN layers become deeper, adjacent node features tends to be similar, making it more difficult for GNNs to distinguish adjacent nodes, thereby, limiting the performance of GNNs. In this paper, we propose conditional local feature encoding (CLFE) to help prevent the problem of node features being dominated by the information from local neighbourhood. The idea of our method is to extract the node hidden state embedding from message passing process and concatenate it with the nodes feature from previous stage, then we utilise linear transformation to form a CLFE based on the concatenated vector. The CLFE will form the layer output to better preserve node-specific information, thus help to improve the performance of GNN models. To verify the feasibility of our method, we conducted extensive experiments on seven benchmark datasets for four graph domain tasks: super-pixel graph classification, node classification, link prediction, and graph regression. The experimental results consistently demonstrate that our method improves model performance across a variety of baseline GNN models for all four tasks.
Paper Structure (9 sections, 4 equations, 2 figures, 8 tables)

This paper contains 9 sections, 4 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Preliminary
  5. The Proposed Conditional Local Feature Encoding
  6. Experiments
  7. Experimental Setup
  8. Experimental Results
  9. CONCLUSION

Figures (2)

  • Figure 1: Proposed method combined with Layer $l+1$ from general GNN model, left hand side represents a vanilla GNN layer with skip connection, and the right hand side of the figure illustrates the GNN layer combined with our method. The red dotted block contains the CLFE plugin component in each GNN layer.
  • Figure 2: Samples of images (first row) and their corresponding superpixel graphs (second row) from MNIST and CIFAR10. The feature of a node consists of the superpixel's coordinate and intensity.