Sequential Recommendation on Temporal Proximities with Contrastive Learning and Self-Attention

TL;DR

TemProxRec addresses the underutilization of temporal context in sequential recommendation by modeling both vertical temporal proximity (across users) and horizontal temporal proximity (within a user sequence). It introduces Temporal-proximity-aware Contrastive Learning (TCL) to align item representations across concurrent users and Multi-Head Absolute-Relative (MHAR) attention to encode time and order within a single user’s history, trained with the MLM objective and TCL loss ${\mathcal{L}_{MLM}}$ and ${\mathcal{L}_{TCL}}$. Empirical results on four real-world datasets show TemProxRec achieving state-of-the-art performance over strong baselines, with ablations confirming the distinct value of TCL and MHAR components. The work demonstrates that explicit temporal proximity modeling improves recommendation quality and offers a plug-in TCL approach that can enhance other transformer-based sequential recommenders, with potential extensions to additional contextual signals.

Abstract

Sequential recommender systems identify user preferences from their past interactions to predict subsequent items optimally. Although traditional deep-learning-based models and modern transformer-based models in previous studies capture unidirectional and bidirectional patterns within user-item interactions, the importance of temporal contexts, such as individual behavioral and societal trend patterns, remains underexplored. Notably, recent models often neglect similarities in users' actions that occur implicitly among users during analogous timeframes-a concept we term vertical temporal proximity. These models primarily adapt the self-attention mechanisms of the transformer to consider the temporal context in individual user actions. Meanwhile, this adaptation still remains limited in considering the horizontal temporal proximity within item interactions, like distinguishing between subsequent item purchases within a week versus a month. To address these gaps, we propose a sequential recommendation model called TemProxRec, which includes contrastive learning and self-attention methods to consider temporal proximities both across and within user-item interactions. The proposed contrastive learning method learns representations of items selected in close temporal periods across different users to be close. Simultaneously, the proposed self-attention mechanism encodes temporal and positional contexts in a user sequence using both absolute and relative embeddings. This way, our TemProxRec accurately predicts the relevant items based on the user-item interactions within a specific timeframe. We validate this work through comprehensive experiments on TemProxRec, consistently outperforming existing models on benchmark datasets as well as showing the significance of considering the vertical and horizontal temporal proximities into sequential recommendation.

Figures (9)

• Figure 1: (As-Is) Recommending lipstick by focusing solely on items within the user's history, (To-Be) Recommending sunscreen and lipstick by considering both items across and within users' history.
• Figure 2: (1) Time interval distribution of sequential item pairs in Amazon Beauty dataset (except for the interval of zero), where each color indicates an item pair (2) Average item overlap ratios of four benchmark datasets
• Figure 3: Overview of TemProxRec. TemProxRec produces item representations from MHAR attention-based transformer layers. Using the final representations, TemProxRec conducts the TCL and the MLM. During the training, TemProxRec (A) performs attention within each sequence for horizontal temporal proximity, (B) contrasts items across sequences for vertical temporal proximity.
• Figure 4: Structure of the proposed Multi-Head Absolute-Relative Attention (A). Self-attention mechanism with absolute embeddings ((A)-1). Self-attention mechanism with relative embeddings ((A)-2).
• Figure 5: Illustration of the sampling strategy for temporal-proximity-aware contrastive learning. The positive samples for an anchor are determined based on vertical temporal proximity which is accessed by a predefined time window.