Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Future Sight and Tough Fights: Revolutionizing Sequential Recommendation with FENRec

Yu-Hsuan Huang, Ling Lo, Hongxia Xie, Hong-Han Shuai, Wen-Huang Cheng

TL;DR

FENRec tackles data sparsity in sequential recommendation by jointly leveraging future interactions through Time-Dependent Soft Labeling and sustaining hard negatives via Enduring Hard Negatives. The method integrates these components into a multi-task learning framework with a revised cross-entropy loss and a contrastive loss that upweights challenging negatives, yielding state-of-the-art results across four benchmark datasets. Empirical analyses show robustness across sequence lengths, improved discriminative representation via item uniformity, and favorable compatibility with existing SR frameworks. The work advances practical SR by more effectively exploiting sparse data and refining the learning signal from both future behavior and hard negatives, with potential extensions to incorporate richer temporal information.

Abstract

Sequential recommendation (SR) systems predict user preferences by analyzing time-ordered interaction sequences. A common challenge for SR is data sparsity, as users typically interact with only a limited number of items. While contrastive learning has been employed in previous approaches to address the challenges, these methods often adopt binary labels, missing finer patterns and overlooking detailed information in subsequent behaviors of users. Additionally, they rely on random sampling to select negatives in contrastive learning, which may not yield sufficiently hard negatives during later training stages. In this paper, we propose Future data utilization with Enduring Negatives for contrastive learning in sequential Recommendation (FENRec). Our approach aims to leverage future data with time-dependent soft labels and generate enduring hard negatives from existing data, thereby enhancing the effectiveness in tackling data sparsity. Experiment results demonstrate our state-of-the-art performance across four benchmark datasets, with an average improvement of 6.16\% across all metrics.

Future Sight and Tough Fights: Revolutionizing Sequential Recommendation with FENRec

TL;DR

FENRec tackles data sparsity in sequential recommendation by jointly leveraging future interactions through Time-Dependent Soft Labeling and sustaining hard negatives via Enduring Hard Negatives. The method integrates these components into a multi-task learning framework with a revised cross-entropy loss and a contrastive loss that upweights challenging negatives, yielding state-of-the-art results across four benchmark datasets. Empirical analyses show robustness across sequence lengths, improved discriminative representation via item uniformity, and favorable compatibility with existing SR frameworks. The work advances practical SR by more effectively exploiting sparse data and refining the learning signal from both future behavior and hard negatives, with potential extensions to incorporate richer temporal information.

Abstract

Sequential recommendation (SR) systems predict user preferences by analyzing time-ordered interaction sequences. A common challenge for SR is data sparsity, as users typically interact with only a limited number of items. While contrastive learning has been employed in previous approaches to address the challenges, these methods often adopt binary labels, missing finer patterns and overlooking detailed information in subsequent behaviors of users. Additionally, they rely on random sampling to select negatives in contrastive learning, which may not yield sufficiently hard negatives during later training stages. In this paper, we propose Future data utilization with Enduring Negatives for contrastive learning in sequential Recommendation (FENRec). Our approach aims to leverage future data with time-dependent soft labels and generate enduring hard negatives from existing data, thereby enhancing the effectiveness in tackling data sparsity. Experiment results demonstrate our state-of-the-art performance across four benchmark datasets, with an average improvement of 6.16\% across all metrics.

Paper Structure

This paper contains 38 sections, 2 theorems, 36 equations, 17 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Sequential Recommendation
  4. Soft Label for Recommendation Systems
  5. Hard Negative Mining in Deep Metrics Learning
  6. Preliminaries
  7. Sequential Recommendation
  8. Contrastive Learning in SR
  9. Method
  10. Future Interaction Utilization
  11. Time-Dependent Soft Labeling.
  12. Enduring Hard Negatives Incorporation
  13. Enduring Hard Negatives.
  14. Hard Negative Upweighting Contrastive Loss.
  15. Multi-Task Learning
  16. ...and 23 more sections

Key Result

Lemma 1

Let $\mathbf{x}$ and $\mathbf{y}$ be non-zero vectors (i.e., $\|\mathbf{x}\|_2 \neq 0$, $\|\mathbf{y}\|_2 \neq 0$) and $\mathbf{x} \neq -\mathbf{y}$. Define $\mathbf{z} = \mathbf{x} + \mathbf{y}$. Then the cosine of the angle between $\mathbf{x}$ and $\mathbf{z}$ is greater than or equal to the cosi

Figures (17)

  • Figure 1: An illustration of binary labels compared to the Time-Dependent Soft Labeling we propose.
  • Figure 2: epoch 5
  • Figure 3: epoch 25
  • Figure 4: epoch 50
  • Figure 6: The framework of our method, FENRec, the user sequences will first be split into subsequences and encoded into representations, while soft labels are generated based on the subsequences. Next, the enduring hard negatives are produced and incorporated into contrastive learning framework. Finally, $L_{rec}^{\prime}$ and $L_{cl}^{\prime}$ are calculated.
  • ...and 12 more figures

Theorems & Definitions (4)

  • Lemma 1
  • proof
  • Lemma 2
  • proof