Learning Positional Attention for Sequential Recommendation

TL;DR

This study delve into the learned positional embedding, demonstrating that it often captures the distance between tokens, and introduces novel attention models that directly learn positional relations.

Abstract

Self-attention-based networks have achieved remarkable performance in sequential recommendation tasks. A crucial component of these models is positional encoding. In this study, we delve into the learned positional embedding, demonstrating that it often captures the distance between tokens. Building on this insight, we introduce novel attention models that directly learn positional relations. Extensive experiments reveal that our proposed models, \textbf{PARec} and \textbf{FPARec} outperform previous self-attention-based approaches. The code can be found here: https://github.com/NetEase-Media/FPARec.

Figures (7)

• Figure 1: maximum input sequence length = 50
• Figure 2: maximum input sequence length = 200
• Figure 4:
• Figure 5:
• Figure 6: