Neighbour-level Message Interaction Encoding for Improved Representation Learning on Graphs

TL;DR

The paper tackles information loss in graph representation learning arising from neglecting neighbour-level message interactions in standard message passing. It introduces a generic neighbour-level message interaction encoding that, at each MP layer, learns encodings between each neighbour's message and the aggregated messages from the rest of the neighborhood, then combines these encodings with the aggregated message to update node embeddings. The method is validated by integrating with GCN and GatedGCN across six benchmark datasets spanning graph classification, node classification, link prediction, and graph regression, achieving improvements and surpassing several baselines. This approach offers a scalable and architecture-agnostic enhancement to deep graph neural networks, enabling more expressive representations for a wide range of graph-structured data.

Abstract

Message passing has become the dominant framework in graph representation learning. The essential idea of the message-passing framework is to update node embeddings based on the information aggregated from local neighbours. However, most existing aggregation methods have not encoded neighbour-level message interactions into the aggregated message, resulting in an information lost in embedding generation. And this information lost could be accumulated and become more serious as more layers are added to the graph network model. To address this issue, we propose a neighbour-level message interaction information encoding method for improving graph representation learning. For messages that are aggregated at a node, we explicitly generate an encoding between each message and the rest messages using an encoding function. Then we aggregate these learned encodings and take the sum of the aggregated encoding and the aggregated message to update the embedding for the node. By this way, neighbour-level message interaction information is integrated into the generated node embeddings. The proposed encoding method is a generic method which can be integrated into message-passing graph convolutional networks. Extensive experiments are conducted on six popular benchmark datasets across four highly-demanded tasks. The results show that integrating neighbour-level message interactions achieves improved performance of the base models, advancing the state of the art results for representation learning over graphs.

Figures (3)

• Figure 1: For each message that passes from a neighbour node to the target node, an encoding is learned between this message and the aggregated message from the rest neighbours. These learned encodings are aggregated to generate a neighbour-level message interaction encoding. The sum of the aggregated message and neighbour-level message encoding is taken to update the embedding for the target node.
• Figure 2: Examples of superpixel graphs generated from the original images in MNIST and CIFAR10. The node features of the superpixel graphs are the coordinates and intensity of the superpixels.
• Figure 3: Performance our model using different graph convolutional layers.