Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Auto-Distillation and Generative Self-Supervised Learning in Residual Graph Transformers for Enhanced Recommender Systems

Eya Mhedhbi, Youssef Mourchid, Alice Othmani

TL;DR

This paper introduces a cutting-edge method for enhancing recommender systems through the integration of generative self-supervised learning (SSL) with a Residual Graph Transformer that consistently outperforms baseline methods.

Abstract

This paper introduces a cutting-edge method for enhancing recommender systems through the integration of generative self-supervised learning (SSL) with a Residual Graph Transformer. Our approach emphasizes the importance of superior data enhancement through the use of pertinent pretext tasks, automated through rationale-aware SSL to distill clear ways of how users and items interact. The Residual Graph Transformer incorporates a topology-aware transformer for global context and employs residual connections to improve graph representation learning. Additionally, an auto-distillation process refines self-supervised signals to uncover consistent collaborative rationales. Experimental evaluations on multiple datasets demonstrate that our approach consistently outperforms baseline methods.

Leveraging Auto-Distillation and Generative Self-Supervised Learning in Residual Graph Transformers for Enhanced Recommender Systems

TL;DR

This paper introduces a cutting-edge method for enhancing recommender systems through the integration of generative self-supervised learning (SSL) with a Residual Graph Transformer that consistently outperforms baseline methods.

Abstract

This paper introduces a cutting-edge method for enhancing recommender systems through the integration of generative self-supervised learning (SSL) with a Residual Graph Transformer. Our approach emphasizes the importance of superior data enhancement through the use of pertinent pretext tasks, automated through rationale-aware SSL to distill clear ways of how users and items interact. The Residual Graph Transformer incorporates a topology-aware transformer for global context and employs residual connections to improve graph representation learning. Additionally, an auto-distillation process refines self-supervised signals to uncover consistent collaborative rationales. Experimental evaluations on multiple datasets demonstrate that our approach consistently outperforms baseline methods.

Paper Structure

This paper contains 16 sections, 13 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Proposed Approach
  3. Problem Formulation
  4. Integration of Global Topology Information
  5. Residual Graph Transformer-Based Collaborative Rationale Discovery
  6. Self-Augmentation with Rationale Awareness
  7. Auto Distillation Graph
  8. Our Objective Loss
  9. Experiments and results
  10. Experimental Settings
  11. Datasets.
  12. Evaluation Protocols.
  13. Hyperparameter Settings
  14. Ablation Study
  15. Comparison with existing methods
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Overall framework illustration of our proposed approach. The figure is better seen zoomed in the digital version of the document.
  • Figure 2: Effect of different components of our proposed models on LastFM dataset in terms of Recall@40 and NDCG@40
  • Figure 3: Effect of different losses used in our model on the Recall@40 and NDCG@40 metrics for the LastFM dataset