Re4: Learning to Re-contrast, Re-attend, Re-construct for Multi-interest Recommendation

TL;DR

This work tackles the challenge of modeling multiple user interests in recommender systems by introducing Re4, a forward-backward learning framework. The forward flow extracts multiple interest embeddings from user behavior via attention, while three backward flows—Re-contrast, Re-attend, and Re-construct—regularize distinctness, align attention with final matching, and semantically reflect representative items. Empirical results on three real-world datasets show Re4 consistently outperforms state-of-the-art multi-interest methods, with substantial gains on larger item galleries, and analyses reveal the backward flows improve both representation quality and ranking performance. The forward-backward paradigm offers a practical, scalable enhancement for learning nuanced, multi-faceted user representations in matching-based recommendation systems.

Abstract

Effectively representing users lie at the core of modern recommender systems. Since users' interests naturally exhibit multiple aspects, it is of increasing interest to develop multi-interest frameworks for recommendation, rather than represent each user with an overall embedding. Despite their effectiveness, existing methods solely exploit the encoder (the forward flow) to represent multiple aspects of interests. However, without explicit regularization, the interest embeddings may not be distinct from each other nor semantically reflect representative historical items. Towards this end, we propose the Re4 framework, which leverages the backward flow to reexamine each interest embedding. Specifically, Re4 encapsulates three backward flows, i.e., 1) Re-contrast, which drives each interest embedding to be distinct from other interests using contrastive learning; 2) Re-attend, which ensures the interest-item correlation estimation in the forward flow to be consistent with the criterion used in final recommendation; and 3) Re-construct, which ensures that each interest embedding can semantically reflect the information of representative items that relate to the corresponding interest. We demonstrate the novel forward-backward multi-interest paradigm on ComiRec, and perform extensive experiments on three real-world datasets. Empirical studies validate that Re4 helps to learn learning distinct and effective multi-interest representations.

Figures (4)

• Figure 1: An illustration of leveraging backward flows for multi-interest representation learning. (a) The traditional forward flow that clusters items and extracts multiple interests. (b) The proposed backward flows, i.e., Re-contrast which learns distinct multi-interests; Re-construct which permits interests' semantic reflection on representative items; and Re-attend which ensures the consistency between attention weights in the forward flow and recommendation correlation.
• Figure 2: Schema of the Re-contrast backward flow, which aims to learn distinct multi-interest representations.
• Figure 3: Test performance across different epochs of Base without backward flow and Re4.
• Figure 4: The visualization displays the multi-interests ($\star$) of some randomly sampled test users, and some corresponding items ($\bullet$ of the same color). We perform t-SNE transformation on the multi-interest embeddings and item embeddings learned by the base model without backward flow and Re4.