Lightweight yet Efficient: An External Attentive Graph Convolutional Network with Positional Prompts for Sequential Recommendation

TL;DR

This work tackles the high computational cost and limited positional modeling of graph-based sequential recommenders by proposing EA-GPS, a lightweight framework that combines a primary graph encoder with a parallel External Attentive Graph Convolutional Network (EA-GCN) and a positional prompt-based decoder. The external encoder uses two memory units to capture global item correlations with near-linear complexity, while the decoder injects absolute item positions as prompts and employs length-adaptive masking plus a soft attention mechanism to capture long-range positional dependencies. Extensive experiments on five real-world datasets show that EA-GPS achieves state-of-the-art performance with significantly fewer parameters and lower training overhead than Transformer-based and other graph-based SRs, demonstrating its practicality for edge devices. The results underscore the value of explicit global weighting via external attention and the effectiveness of prompt-based positional modeling for user-specific sequence representations.

Abstract

Graph-based Sequential Recommender systems (GSRs) have gained significant research attention due to their ability to simultaneously handle user-item interactions and sequential relationships between items. Current GSRs often utilize composite or in-depth structures for graph encoding (e.g., the Graph Transformer). Nevertheless, they have high computational complexity, hindering the deployment on resource-constrained edge devices. Moreover, the relative position encoding in Graph Transformer has difficulty in considering the complicated positional dependencies within sequence. To this end, we propose an External Attentive Graph convolutional network with Positional prompts for Sequential recommendation, namely EA-GPS. Specifically, we first introduce an external attentive graph convolutional network that linearly measures the global associations among nodes via two external memory units. Then, we present a positional prompt-based decoder that explicitly treats the absolute item positions as external prompts. By introducing length-adaptive sequential masking and a soft attention network, such a decoder facilitates the model to capture the long-term positional dependencies and contextual relationships within sequences. Extensive experimental results on five real-world datasets demonstrate that the proposed EA-GPS outperforms the state-of-the-art methods. Remarkably, it achieves the superior performance while maintaining a smaller parameter size and lower training overhead. The implementation of this work is publicly available at https://github.com/ZZY-GraphMiningLab/EA-GPS.

Figures (8)

• Figure 1: An example of the sequential graph, where user-item interactions are denoted by red arrows and sequential dependencies between items are in blue.
• Figure 2: Framework of EA-GPS, where $u_1$ and $u_2$ are two different users, and $\{i_1, i_2, \dots, i_6\}$ are the interacted items that compose the behavioral sequences for these users.
• Figure 3: The differences between Self-Attention (SA), Linear Attention (LA) and External Attention (EA).
• Figure 4: The architecture of the positional prompt-based decoder (taking $S_k$ as an example).
• Figure 5: Time consumption and the parameter scale of EA-GPS compared with LightGCN, TGT, PTGCN and GCL4SR.