Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Equivariant Contrastive Learning for Sequential Recommendation

Peilin Zhou, Jingqi Gao, Yueqi Xie, Qichen Ye, Yining Hua, Jae Boum Kim, Shoujin Wang, Sunghun Kim

TL;DR

This work proposes Equivariant Contrastive Learning for Sequential Recommendation (ECL-SR), which endows SR models with great discriminative power, making the learned user behavior representations sensitive to invasive augmentations and insensitive to mild augmentations.

Abstract

Contrastive learning (CL) benefits the training of sequential recommendation models with informative self-supervision signals. Existing solutions apply general sequential data augmentation strategies to generate positive pairs and encourage their representations to be invariant. However, due to the inherent properties of user behavior sequences, some augmentation strategies, such as item substitution, can lead to changes in user intent. Learning indiscriminately invariant representations for all augmentation strategies might be suboptimal. Therefore, we propose Equivariant Contrastive Learning for Sequential Recommendation (ECL-SR), which endows SR models with great discriminative power, making the learned user behavior representations sensitive to invasive augmentations (e.g., item substitution) and insensitive to mild augmentations (e.g., featurelevel dropout masking). In detail, we use the conditional discriminator to capture differences in behavior due to item substitution, which encourages the user behavior encoder to be equivariant to invasive augmentations. Comprehensive experiments on four benchmark datasets show that the proposed ECL-SR framework achieves competitive performance compared to state-of-the-art SR models. The source code is available at https://github.com/Tokkiu/ECL.

Equivariant Contrastive Learning for Sequential Recommendation

TL;DR

This work proposes Equivariant Contrastive Learning for Sequential Recommendation (ECL-SR), which endows SR models with great discriminative power, making the learned user behavior representations sensitive to invasive augmentations and insensitive to mild augmentations.

Abstract

Contrastive learning (CL) benefits the training of sequential recommendation models with informative self-supervision signals. Existing solutions apply general sequential data augmentation strategies to generate positive pairs and encourage their representations to be invariant. However, due to the inherent properties of user behavior sequences, some augmentation strategies, such as item substitution, can lead to changes in user intent. Learning indiscriminately invariant representations for all augmentation strategies might be suboptimal. Therefore, we propose Equivariant Contrastive Learning for Sequential Recommendation (ECL-SR), which endows SR models with great discriminative power, making the learned user behavior representations sensitive to invasive augmentations (e.g., item substitution) and insensitive to mild augmentations (e.g., featurelevel dropout masking). In detail, we use the conditional discriminator to capture differences in behavior due to item substitution, which encourages the user behavior encoder to be equivariant to invasive augmentations. Comprehensive experiments on four benchmark datasets show that the proposed ECL-SR framework achieves competitive performance compared to state-of-the-art SR models. The source code is available at https://github.com/Tokkiu/ECL.
Paper Structure (33 sections, 8 equations, 5 figures, 5 tables)

This paper contains 33 sections, 8 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. PRELIMINARIES
  3. Problem Formulation
  4. Data Augmentation Strategies
  5. Invariant Contrastive Learning for Sequential Recommendation
  6. Methodology
  7. Equivariant Contrastive Learning for Sequential Recommendation
  8. Learning Invariance for Mild Augmentation
  9. Learning Equivariance for Invasive Augmentation
  10. Optimization
  11. Model Training and Inference
  12. Model Complexity
  13. Experiment
  14. Experimental Settings
  15. Dataset.
  16. ...and 18 more sections

Figures (5)

  • Figure 1: (a) An example of invasive data augmentation; (b) An example of mild augmentation; (c) Performance comparison of using different data augmentations to generate positive pairs for invariant contrastive learning on Yelp dataset.
  • Figure 2: The connection between invariant contrastive learning (ICL) and equivariant contrastive learning (ECL): ECL is a generalization of traditional ICL methods that highlights the complementary nature of invariance and equivariance.
  • Figure 3: Overview of the proposed ECL-SR framework.
  • Figure 4: Ablation study of ECL-SR on ML-1m and Toys datasets.
  • Figure 5: Performance (Recall@10) comparison with respect to 5 hyper-parameters on Toys and Yelp datasets.