Improving Graph Convolutional Networks with Transformer Layer in social-based items recommendation

TL;DR

The paper tackles rating prediction on social graphs by augmenting a Graph Convolutional Network with a Transformer Encoder to enable attentive, multi-hop aggregation. The Graph Transformer Network GTN first learns node embeddings with two GCN layers and then refines them with self-attention, learning representations that capture social and item interactions without exhaustive hand-tuning. Datasets from Ciao and Epinions show GTN consistently outperforms PMF, NeuMF, GraphRec, and plain GCN, indicating that attention-based reweighting of neighbor information improves link prediction. The work highlights the practical value of integrating Transformer-style attention into graph embeddings for social-based recommendations and provides a code baseline for future research.

Abstract

In this work, we have proposed an approach for improving the GCN for predicting ratings in social networks. Our model is expanded from the standard model with several layers of transformer architecture. The main focus of the paper is on the encoder architecture for node embedding in the network. Using the embedding layer from the graph-based convolution layer, the attention mechanism could rearrange the feature space to get a more efficient embedding for the downstream task. The experiments showed that our proposed architecture achieves better performance than GCN on the traditional link prediction task.

Figures (2)

• Figure 1: Illustration of the architecture of the proposed model
• Figure 2: Loss in the training process with our model and baseline models on Ciao dataset