DiffCL: A Diffusion-Based Contrastive Learning Framework with Semantic Alignment for Multimodal Recommendations

TL;DR

DiffCL tackles data sparsity and multimodal noise in recommender systems by integrating diffusion-based contrastive learning with a graph-based multimodal framework. It uses a Graph Encoder to extract higher-order cues across visual, textual, and ID graphs, then applies a diffusion forward and reverse process to generate informative contrastive views, optimized with an InfoNCE-based loss. It further enhances modality-specific representations via an Item-Item graph and aligns semantic information across modalities using stable ID embeddings, with a joint optimization including BPR, diffusion contrastive loss, and alignment loss. Empirical results on three public datasets show that DiffCL consistently improves over state-of-the-art baselines, validating the effectiveness of diffusion-based view construction and ID-guided semantic alignment for multimodal recommendations.

Abstract

Multimodal recommendation systems integrate diverse multimodal information into the feature representations of both items and users, thereby enabling a more comprehensive modeling of user preferences. However, existing methods are hindered by data sparsity and the inherent noise within multimodal data, which impedes the accurate capture of users' interest preferences. Additionally, discrepancies in the semantic representations of items across different modalities can adversely impact the prediction accuracy of recommendation models. To address these challenges, we introduce a novel diffusion-based contrastive learning framework (DiffCL) for multimodal recommendation. DiffCL employs a diffusion model to generate contrastive views that effectively mitigate the impact of noise during the contrastive learning phase. Furthermore, it improves semantic consistency across modalities by aligning distinct visual and textual semantic information through stable ID embeddings. Finally, the introduction of the Item-Item Graph enhances multimodal feature representations, thereby alleviating the adverse effects of data sparsity on the overall system performance. We conduct extensive experiments on three public datasets, and the results demonstrate the superiority and effectiveness of the DiffCL.

Figures (5)

• Figure 1: Two methods for constructing graph contrastive learning: edge dropout and add random noise. (a) Some edges are randomly selected and removed from the graph according to a predefined dropout rate. (b) Add random uniform noise or gaussian noise to the feature embeddings after they have been processed by the Graph Encoder.
• Figure 2: The overview architecture of our DiffCL framework. The DiffCL consists of three modules. (a)Graph Encoder is used to extract higher-order user preference cues and collaborative signals. The $G=\left \{G_{m} \mid G_{v} , G_{t}, G_{id} \right \}$ represents three different user-item graphs. The $E_{m}$ are the feature embeddings of different modalities obtained by GCNs, $m\in \left \{v, t, id \right \}$. (b)Diffusion Graph Contrastive introduces diffusion model to construct contrast views for contrastive learning. $E^{1}_{m}$ and $E^{2}_{m}$ are feature embeddings obtained by diffusion model, $m\in \left \{v, t \right \}$. (c)Enhancement and Alignment implements item semantic enhancement and cross-modal semantic alignment. The ID embedding includes $E^{u}_{id}$ and $E^{i}_{id}$.
• Figure 3: The performance of the DiffCL under various $\lambda_{diff}$ settings
• Figure 4: The performance of the DiffCL under various $\lambda_{align}$ settings
• Figure 5: The performance of the DiffCL under various $\lambda_{E}$ settings