GrIT: Group Informed Transformer for Sequential Recommendation

TL;DR

This work tackles sequential recommendation by incorporating temporally evolving group affiliations into user representations. It introduces GrIT, a transformer-based model that jointly models individual interaction sequences and latent group dynamics through time-varying membership weights learned from short- and long-term windows, including a transition-sequence component. Key contributions include a four-step group representation pipeline—transition construction, temporal representations with EWMA, group affinity via softmax with temperature, and group-aware representation learning—and a fusion mechanism with personal sequence signals, all trained end-to-end. Experiments on five benchmarks show consistent improvements in Recall@$k$, NDCG@$k$, and MRR@$k$ over state-of-the-art baselines, underscoring the value of modeling evolving group context for context-aware next-item prediction.

Abstract

Sequential recommender systems aim to predict a user's future interests by extracting temporal patterns from their behavioral history. Existing approaches typically employ transformer-based architectures to process long sequences of user interactions, capturing preference shifts by modeling temporal relationships between items. However, these methods often overlook the influence of group-level features that capture the collective behavior of similar users. We hypothesize that explicitly modeling temporally evolving group features alongside individual user histories can significantly enhance next-item recommendation. Our approach introduces latent group representations, where each user's affiliation to these groups is modeled through learnable, time-varying membership weights. The membership weights at each timestep are computed by modeling shifts in user preferences through their interaction history, where we incorporate both short-term and long-term user preferences. We extract a set of statistical features that capture the dynamics of user behavior and further refine them through a series of transformations to produce the final drift-aware membership weights. A group-based representation is derived by weighting latent group embeddings with the learned membership scores. This representation is integrated with the user's sequential representation within the transformer block to jointly capture personal and group-level temporal dynamics, producing richer embeddings that lead to more accurate, context-aware recommendations. We validate the effectiveness of our approach through extensive experiments on five benchmark datasets, where it consistently outperforms state-of-the-art sequential recommendation methods.

Figures (9)

• Figure 1: Overview of GrIT.
• Figure 2: Performance of comparing algorithms in terms of $MRR@k$.
• Figure 3: Influence of positional encoding strategies.
• Figure 4: Effect of features on GrIT architecture.
• Figure 5: Cosine similarity between groups.