Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sequential Recommendation via Adaptive Robust Attention with Multi-dimensional Embeddings

Linsey Pang, Amir Hossein Raffiee, Wei Liu, Keld Lundgaard

TL;DR

This study improves the sequential recommender model’s robustness and generalization by introducing a mix-attention mechanism with a layer-wise noise injection (LNI) regularization and demonstrates that the model outperforms existing self-attention architectures.

Abstract

Sequential recommendation models have achieved state-of-the-art performance using self-attention mechanism. It has since been found that moving beyond only using item ID and positional embeddings leads to a significant accuracy boost when predicting the next item. In recent literature, it was reported that a multi-dimensional kernel embedding with temporal contextual kernels to capture users' diverse behavioral patterns results in a substantial performance improvement. In this study, we further improve the sequential recommender model's robustness and generalization by introducing a mix-attention mechanism with a layer-wise noise injection (LNI) regularization. We refer to our proposed model as adaptive robust sequential recommendation framework (ADRRec), and demonstrate through extensive experiments that our model outperforms existing self-attention architectures.

Sequential Recommendation via Adaptive Robust Attention with Multi-dimensional Embeddings

TL;DR

This study improves the sequential recommender model’s robustness and generalization by introducing a mix-attention mechanism with a layer-wise noise injection (LNI) regularization and demonstrates that the model outperforms existing self-attention architectures.

Abstract

Sequential recommendation models have achieved state-of-the-art performance using self-attention mechanism. It has since been found that moving beyond only using item ID and positional embeddings leads to a significant accuracy boost when predicting the next item. In recent literature, it was reported that a multi-dimensional kernel embedding with temporal contextual kernels to capture users' diverse behavioral patterns results in a substantial performance improvement. In this study, we further improve the sequential recommender model's robustness and generalization by introducing a mix-attention mechanism with a layer-wise noise injection (LNI) regularization. We refer to our proposed model as adaptive robust sequential recommendation framework (ADRRec), and demonstrate through extensive experiments that our model outperforms existing self-attention architectures.
Paper Structure (13 sections, 1 figure, 6 tables, 1 algorithm)

This paper contains 13 sections, 1 figure, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Our Learning Framework
  4. Problem Formulation
  5. Input Representation
  6. Attention Mechanism
  7. Robust Input and Output
  8. Learning Objective
  9. Experiments
  10. Comparison with Baseline Models
  11. Robustness Analysis
  12. Ablation Studies
  13. Conclusion

Figures (1)

  • Figure 1: (a) Review trend for a perfume product, showing an increase in reviews from 2005 to 2014 with growing popularity. (b) Review trend for another product, showing a decline in reviews from 2005 to 2014, possibly indicating reduced consumer interest. (c) User's interest in face masks. (d) User's review distribution reveals consistent purchasing behavior for facial masks every 3 months, mainly choosing the Boscia brand.(Data Source: Amazon Beauty Review kang2018self)