Improving Sequential Recommendations via Bidirectional Temporal Data Augmentation with Pre-training

TL;DR

BARec tackles the challenge of learning from short sequences in sequential recommendations by introducing bidirectional temporal data augmentation with pre-training and a knowledge-enhanced fine-tuning stage. The method generates high-quality pseudo-prior items via reverse generation aligned to forward preferences, and decouples augmentation benefits from representations using a bidirectional KL-divergence objective. Empirical results across five benchmarks and large-scale Tenrec show clear gains over strong baselines, with theoretical backing confirming preference preservation and interpretable improvements. The work advances practical SR by improving performance on both very short and very long sequences and offers a clear path toward integrating more powerful generative architectures in the future.

Abstract

Sequential recommendation systems are integral to discerning temporal user preferences. Yet, the task of learning from abbreviated user interaction sequences poses a notable challenge. Data augmentation has been identified as a potent strategy to enhance the informational richness of these sequences. Traditional augmentation techniques, such as item randomization, may disrupt the inherent temporal dynamics. Although recent advancements in reverse chronological pseudo-item generation have shown promise, they can introduce temporal discrepancies when assessed in a natural chronological context. In response, we introduce a sophisticated approach, Bidirectional temporal data Augmentation with pre-training (BARec). Our approach leverages bidirectional temporal augmentation and knowledge-enhanced fine-tuning to synthesize authentic pseudo-prior items that retain user preferences and capture deeper item semantic correlations, thus boosting the model's expressive power. Our comprehensive experimental analysis on five benchmark datasets confirms the superiority of BARec across both short and elongated sequence contexts. Moreover, theoretical examination and case study offer further insight into the model's logical processes and interpretability. The source code for our study is publicly available at https://github.com/juyongjiang/BARec.

Figures (5)

• Figure 1: Performance (Recall@5) w.r.t sequence length distribution (bar) on Amazon Beauty when using (1) no sequence augmentation (SASRec, purple dots); (2) reverse sequence augmentation (ASReP, green squares) and (3) our bidirectional temporal augmentation (BARec, yellow stars, being particularly advantageous on short sequences). In the case of long sequences with $\ge 20$, ASReP either matches or underperforms relative to SASRec, as shown in red dotted rectangle.
• Figure 2: The architecture of our proposed BARec framework encompasses three core components: (a) a model-agnostic backbone featuring an embedding layer, $L\times$ Encoder layers, and a prediction layer; (b) a bidirectional temporal data augmentation strategy, coupled with pre-training, designed to generate high-fidelity pseudo-prior items that retain user preferences and enhance the semantic interrelations among items; (c) a knowledge-enhanced fine-tuning phase tailored for downstream tasks, specifically, sequential recommendation.
• Figure 3: Performance comparison of a variety of data augmentation strategies for sequential recommendation. The relative improvements over baseline SASRec kang2018self with Recall@10 metric on Beauty (left side) and Phones (right side) are reported.
• Figure 4: Comprehensive hyper-parameters sensitivity analysis, including trade-off coefficient $\lambda$, balance coefficient $\alpha$, clipped gold labels $clip_k$, model dimension $d_{model}$, attention heads $h$, batch size $b$, pseudo-prior items $\mathcal{K}$, and sequence length threshold $\mathcal{M}$, as applied to the Beauty dataset.
• Figure 5: A case study on a randomly selected user from Amazon's product data within the Beauty category. The ASReP and BARec methods employ data augmentation strategies to enrich the context of short sequences by generating pseudo-prior items. The results of the top-5 product recommendations using the SASRec, ASReP, and BARec models indicate that our BARec model more accurately captures user preference patterns.