T-REX: Transformer-Based Category Sequence Generation for Grocery Basket Recommendation

TL;DR

T-REX is proposed, a novel transformer architecture that generates personalized category-level suggestions by learning both short-term basket dependencies and long-term user preferences that better aligns with the sequential nature of basket generation, enabling more accurate next-basket predictions.

Abstract

Online grocery shopping presents unique challenges for sequential recommendations due to repetitive purchase patterns and complex item relationships within the baskets. Unlike traditional e-commerce, grocery recommendations must capture both complementary item associations and temporal dependencies across shopping sessions. To address these challenges in Amazon's online grocery business, we propose T-REX, a novel transformer architecture that generates personalized category-level suggestions by learning both short-term basket dependencies and long-term user preferences. Our approach introduces three key innovations: (1) an efficient sampling strategy utilizing dynamic sequence splitting for sparse shopping patterns, (2) an adaptive positional encoding scheme for temporal patterns, and (3) a category-level modeling approach that reduces dimensionality while maintaining recommendation quality. Although masked language modeling techniques like BERT4Rec excel at capturing item relations, they prove less suitable for next basket generation due to information leakage issues. In contrast, T-REX's causal masking approach better aligns with the sequential nature of basket generation, enabling more accurate next-basket predictions. Experiments on large-scale grocery offline data and online A/B tests show significant improvement over existing systems.

Figures (5)

• Figure 1: Illustration of dynamic sequence splitting and corresponding attention mechanisms in T-REX. The example shows a synthetic purchase sequence used for demonstration purposes only.
• Figure 2: Comparison of average recall@k (left) and precision@k (right) between Transformer and P-Top models.
• Figure 3: Variance analysis of recall@k for transformer (left) and P-Top baseline (right). Box-plots present the distribution of Recall at each value of ‘k’. The bottom of each box is the 25 percentile, the orange bar is the median, the top of the box is the 75 percentile and circles are the outliers.
• Figure 4: Breakdown of recall@10 for the number of previous sessions (right) as well as the length of the last basket (left). Results are from 10,000 sub-sample of the test dataset.
• Figure 5: The proposed rank-matching metric, comparing the actual ranking of categories in the customers' last basket with the predicted rankings from the transformer model (left) and rankings from the P-Top baseline (right).