Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering

TL;DR

This work addresses the dimensional collapse observed in contrastive learning for collaborative filtering by proposing nCL, a non-contrastive objective that imposes alignment between positive user-item pairs and a rate-distortion based compactness to preserve the full embedding space. It introduces cluster-informed variants (nCLG and nCL) to capture community structure and uses LightGCN as backbone, enabling scalable training without data augmentation or negative sampling. Empirical results on four public datasets show that nCL consistently outperforms state-of-the-art contrastive methods and improves robustness to sparsity, with favorable training efficiency. The approach offers a practical, scalable alternative for learning high-quality, discriminative user/item representations in large-scale recommender systems, with potential extensions to sequential and fairness-oriented tasks.

Abstract

Contrastive Learning (CL) has shown promising performance in collaborative filtering. The key idea is to generate augmentation-invariant embeddings by maximizing the Mutual Information between different augmented views of the same instance. However, we empirically observe that existing CL models suffer from the \textsl{dimensional collapse} issue, where user/item embeddings only span a low-dimension subspace of the entire feature space. This suppresses other dimensional information and weakens the distinguishability of embeddings. Here we propose a non-contrastive learning objective, named nCL, which explicitly mitigates dimensional collapse of representations in collaborative filtering. Our nCL aims to achieve geometric properties of \textsl{Alignment} and \textsl{Compactness} on the embedding space. In particular, the alignment tries to push together representations of positive-related user-item pairs, while compactness tends to find the optimal coding length of user/item embeddings, subject to a given distortion. More importantly, our nCL does not require data augmentation nor negative sampling during training, making it scalable to large datasets. Experimental results demonstrate the superiority of our nCL.

Figures (6)

• Figure 1: Different collapsing patterns (complete collapse, dimensional collapse, and no collapse) for three item groups.
• Figure 2: Singular value spectrum of all the embeddings. The spectrum contains the singular values of the covariance matrix in sorted order. A number of singular values converge to zero (e.g., LightGCN), indicating dimensional collapse.
• Figure 3: Overview of the proposed nCL. nCL achieves two purposes: 1) alignment that minimizes distances between positive user-item pairs, and 2) compactness that encourages the embeddings of users/items to expand as much volume as possible, while simultaneously pushing each cluster (e.g., squares and circles) to occupy as little space as possible.
• Figure 4: The training curves (testing recall@10) of our loss and the BPR loss for MovieLens10M and Yelp datasets.
• Figure 5: Performance analysis (Recall@10) for different levels of sparsity for MovieLens10M and Yelp datasets.