RecDCL: Dual Contrastive Learning for Recommendation

TL;DR

RecDCL tackles sparse user-item data by jointly optimizing batch-wise contrastive learning (BCL) and feature-wise contrastive learning (FCL) within a dual-contrastive framework. It defines two objective components, $L_{UIBT}$ and $L_{UUII}$, and combines them with a batch-wise loss $L_{BCL}$ as $\\mathcal{\\L} = L_{UIBT} + \alpha L_{UUII} + \beta L_{BCL}$, while standardizing embeddings to reveal the native connection between BCL and FCL. The paper proves that combining BCL and FCL reduces redundant solutions without missing optimal ones and demonstrates robustness and improvements over state-of-the-art methods on four public datasets and one industrial dataset, including up to $5.65\%$ recall and $5.34\%$ NDCG gains. The authors release the public code and show promising practical impact for real-world recommender systems.

Abstract

Self-supervised learning (SSL) has recently achieved great success in mining the user-item interactions for collaborative filtering. As a major paradigm, contrastive learning (CL) based SSL helps address data sparsity in Web platforms by contrasting the embeddings between raw and augmented data. However, existing CL-based methods mostly focus on contrasting in a batch-wise way, failing to exploit potential regularity in the feature dimension. This leads to redundant solutions during the representation learning of users and items. In this work, we investigate how to employ both batch-wise CL (BCL) and feature-wise CL (FCL) for recommendation. We theoretically analyze the relation between BCL and FCL, and find that combining BCL and FCL helps eliminate redundant solutions but never misses an optimal solution. We propose a dual contrastive learning recommendation framework -- RecDCL. In RecDCL, the FCL objective is designed to eliminate redundant solutions on user-item positive pairs and to optimize the uniform distributions within users and items using a polynomial kernel for driving the representations to be orthogonal; The BCL objective is utilized to generate contrastive embeddings on output vectors for enhancing the robustness of the representations. Extensive experiments on four widely-used benchmarks and one industry dataset demonstrate that RecDCL can consistently outperform the state-of-the-art GNNs-based and SSL-based models (with an improvement of up to 5.65\% in terms of Recall@20). The source code is publicly available (https://github.com/THUDM/RecDCL).

Figures (13)

• Figure 1: The motivating example that shows the effect for a negative pair in BCL, FCL, and BCL+FCL, where the light-shaded symbols indicate potentially possible solutions. In this example, BCL (top right) tends to align the negative pair on a straight line, i.e., even distribution in a circle; FCL (bottom left) mostly encourages the two representations to be orthogonal; BCL+FCL (bottom right) drives the two samples to saturate on either the $x$ axis or the $y$ axis. Note that using either BCL alone or FCL alone will result in infinite potential solutions. In contrast, combining BCL and FCL yields only four possible solutions: $\{(0,1),(0,-1)\}$, $\{(0,-1),(0,1)\}$, $\{(1,0),(-1,0)\}$ and $\{(-1,0),(1,0)\}$, which cancels the redundant solutions but never misses an optimal solution, and thus is intuitively a more reasonable regularization compared with BCL alone or FCL alone. High-dimensional cases are analogous.
• Figure 2: The overall framework of RecDCL. In this framework, (a) denotes the cross-correlation matrix to an identity matrix between users and items in FCL; (b) and (c) stand for the distribution uniformity within users and items in FCL; (d) denotes the random distribution on user-item positive pairs in BCL; (e) demonstrates the final distribution on users and items produced by a dual CL.
• Figure 3: The components of Projector.
• Figure 4: Recall@20 results of different embedding sizes of representative baselines and RecDCL on Beauty and Yelp.
• Figure 5: Influence of different $\alpha$ of UUII on Beauty and Yelp.