Cross-Representation Knowledge Transfer for Improved Sequential Recommendations

TL;DR

This work presents a new framework that combines transformers and graph neural networks and aligns different representations for solving next-item prediction task and demonstrates that the proposed framework consistently outperforms both pure sequential and graph approaches in terms of recommendation quality.

Abstract

Transformer architectures, capable of capturing sequential dependencies in the history of user interactions, have become the dominant approach in sequential recommender systems. Despite their success, such models consider sequence elements in isolation, implicitly accounting for the complex relationships between them. Graph neural networks, in contrast, explicitly model these relationships through higher order interactions but are often unable to adequately capture their evolution over time, limiting their use for predicting the next interaction. To fill this gap, we present a new framework that combines transformers and graph neural networks and aligns different representations for solving next-item prediction task. Our solution simultaneously encodes structural dependencies in the interaction graph and tracks their dynamic change. Experimental results on a number of open datasets demonstrate that the proposed framework consistently outperforms both pure sequential and graph approaches in terms of recommendation quality, as well as recent methods that combine both types of signals.

Figures (4)

• Figure 1: Visualized training process, with training loss and NDCG@10 scales placed on the left and on the right accordingly.
• Figure 2: Graph size ablation for MovieLens-1M and Amazon Beauty datasets
• Figure 3: Performance (NDCG@10) vs. the number of GCN warm-up epochs.
• Figure 4: Test ndcg@10 metric by the Barlow coefficient.