Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

InfoDCL: Informative Noise Enhanced Diffusion Based Contrastive Learning

Xufeng Liang, Zhida Qin, Chong Zhang, Tianyu Huang, Gangyi Ding

TL;DR

This work tackles the sparsity of user-item interactions in recommender systems by moving beyond random perturbations in contrastive learning. It introduces InfoDCL, a diffusion-based framework that generates informative noise through a single-step diffusion process guided by auxiliary metadata via PSNet, including spectral rectification and contextual re-encoding. A collaborative training objective aligns generation, contrastive learning, and ranking signals, while keeping the diffusion process off during training and using inference-time LightGCN to inject higher-order structure efficiently. Across five real-world datasets, InfoDCL consistently outperforms state-of-the-art baselines, demonstrating the value of semantically informed diffusion for learning authentic user preferences in sparse settings.

Abstract

Contrastive learning has demonstrated promising potential in recommender systems. Existing methods typically construct sparser views by randomly perturbing the original interaction graph, as they have no idea about the authentic user preferences. Owing to the sparse nature of recommendation data, this paradigm can only capture insufficient semantic information. To address the issue, we propose InfoDCL, a novel diffusion-based contrastive learning framework for recommendation. Rather than injecting randomly sampled Gaussian noise, we employ a single-step diffusion process that integrates noise with auxiliary semantic information to generate signals and feed them to the standard diffusion process to generate authentic user preferences as contrastive views. Besides, based on a comprehensive analysis of the mutual influence between generation and preference learning in InfoDCL, we build a collaborative training objective strategy to transform the interference between them into mutual collaboration. Additionally, we employ multiple GCN layers only during inference stage to incorporate higher-order co-occurrence information while maintaining training efficiency. Extensive experiments on five real-world datasets demonstrate that InfoDCL significantly outperforms state-of-the-art methods. Our InfoDCL offers an effective solution for enhancing recommendation performance and suggests a novel paradigm for applying diffusion method in contrastive learning frameworks.

InfoDCL: Informative Noise Enhanced Diffusion Based Contrastive Learning

TL;DR

This work tackles the sparsity of user-item interactions in recommender systems by moving beyond random perturbations in contrastive learning. It introduces InfoDCL, a diffusion-based framework that generates informative noise through a single-step diffusion process guided by auxiliary metadata via PSNet, including spectral rectification and contextual re-encoding. A collaborative training objective aligns generation, contrastive learning, and ranking signals, while keeping the diffusion process off during training and using inference-time LightGCN to inject higher-order structure efficiently. Across five real-world datasets, InfoDCL consistently outperforms state-of-the-art baselines, demonstrating the value of semantically informed diffusion for learning authentic user preferences in sparse settings.

Abstract

Contrastive learning has demonstrated promising potential in recommender systems. Existing methods typically construct sparser views by randomly perturbing the original interaction graph, as they have no idea about the authentic user preferences. Owing to the sparse nature of recommendation data, this paradigm can only capture insufficient semantic information. To address the issue, we propose InfoDCL, a novel diffusion-based contrastive learning framework for recommendation. Rather than injecting randomly sampled Gaussian noise, we employ a single-step diffusion process that integrates noise with auxiliary semantic information to generate signals and feed them to the standard diffusion process to generate authentic user preferences as contrastive views. Besides, based on a comprehensive analysis of the mutual influence between generation and preference learning in InfoDCL, we build a collaborative training objective strategy to transform the interference between them into mutual collaboration. Additionally, we employ multiple GCN layers only during inference stage to incorporate higher-order co-occurrence information while maintaining training efficiency. Extensive experiments on five real-world datasets demonstrate that InfoDCL significantly outperforms state-of-the-art methods. Our InfoDCL offers an effective solution for enhancing recommendation performance and suggests a novel paradigm for applying diffusion method in contrastive learning frameworks.

Paper Structure

This paper contains 34 sections, 2 theorems, 35 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Methodology
  4. Informative Noise Generation
  5. Auxiliary Metadata Synthesis
  6. Spectral Rectification
  7. Contextual Re-encoding
  8. Semantics Enhanced Contrastive Learning
  9. Collaborative Training Objective Strategy
  10. Inference Stage
  11. Theoretical Analysis
  12. Experiment
  13. Experimental Settings
  14. Datasets
  15. Evaluation Metrics
  16. ...and 19 more sections

Key Result

theorem 1

Let $\boldsymbol{\varepsilon}_{\boldsymbol{\theta}}(\mathbf{v}, t \mid \star)$ be a trained noise prediction network that is $L$-Lipschitz with respect to its first argument. Assume the trajectory is smooth over a step size $k \ll T$, i.e., Let the classifier-free guidance (CFG) scale at step $T$ and $T-k$ be $\boldsymbol{\omega}_{\ell}$ and $\boldsymbol{\omega}_{w}$, respectively. Then the output

Figures (9)

  • Figure 1: The overall architecture of our proposed InfoDCL, which involves informative noise generation by applying SVD to approximate a single-step diffusion process, where randomly sampled Gaussian noise and auxiliary metadata with semantic information are fused to produce informative noise. Then this signal is utilized in the standard diffusion paradigm to generate item embeddings with authentic user preferences. The collaborative training objective strategy continuously optimizes generation, preference learning and contrastive learning tasks. During inference, the GCN is incorporated to enhance representations with higher-order co-occurrence information, improving training efficiency.
  • Figure 2: Ablation analysis across five datasets
  • Figure 3: Performance versus efficiency analysis on Amazon-Electronics.
  • Figure 4: SNR Comparison on Amazon-Baby, Amazon-Electronics and Taobao
  • Figure 5: Visualization of the item embeddings on Amazon-Baby dataset using T-SNE.
  • ...and 4 more figures

Theorems & Definitions (2)

  • theorem 1
  • theorem 2