RAU: Towards Regularized Alignment and Uniformity for Representation Learning in Recommendation

TL;DR

This paper addresses two core challenges in AU-based representation learning for recommender systems: sparse alignment and uneven uniformity caused by data sparsity. It introduces Regularized Alignment and Uniformity (RAU), consisting of Center-strengthened Alignment to reinforce alignment and Low-variance Guided Uniformity to stabilize uniformity growth, combined in a unified objective. Empirical results on three real-world datasets show that RAU consistently outperforms state-of-the-art baselines, including graph-based backbones, with notable gains in Recall@20 and NDCG@20. The findings demonstrate that targeted regularization of representation distribution properties can surpass sophisticated encoder designs, offering encoder-agnostic benefits and practical impact for scalable RecSys deployments.

Abstract

Recommender systems (RecSys) have become essential in modern society, driving user engagement and satisfaction across diverse online platforms. Most RecSys focuses on designing a powerful encoder to embed users and items into high-dimensional vector representation space, with loss functions optimizing their representation distributions. Recent studies reveal that directly optimizing key properties of the representation distribution, such as alignment and uniformity, can outperform complex encoder designs. However, existing methods for optimizing critical attributes overlook the impact of dataset sparsity on the model: limited user-item interactions lead to sparse alignment, while excessive interactions result in uneven uniformity, both of which degrade performance. In this paper, we identify the sparse alignment and uneven uniformity issues, and further propose Regularized Alignment and Uniformity (RAU) to cope with these two issues accordingly. RAU consists of two novel regularization methods for alignment and uniformity to learn better user/item representation. 1) Center-strengthened alignment further aligns the average in-batch user/item representation to provide an enhanced alignment signal and further minimize the disparity between user and item representation. 2) Low-variance-guided uniformity minimizes the variance of pairwise distances along with uniformity, which provides extra guidance to a more stabilized uniformity increase during training. We conducted extensive experiments on three real-world datasets, and the proposed RAU resulted in significant performance improvements compared to current state-of-the-art CF methods, which confirms the advantages of the two proposed regularization methods.

Figures (6)

• Figure 1: Alignment change along the training on different datasets with varied average user/item interaction numbers.
• Figure 2: The sample points are distributed on the unit circle. We find that randomly increasing uniformity does not necessarily achieve uniformity.
• Figure 3: Overview of the RAU framework.
• Figure 4: Performance of different alignment strengths.
• Figure 5: Comparative Analysis of Uniformity Loss and Variance Before and After RU Implementation.