GemiRec: Interest Quantization and Generation for Multi-Interest Recommendation

TL;DR

GemiRec introduces a novel framework for multi-interest recommendation that tackles two core limitations—interest collapse and insufficient modeling of interest evolution—by combining interest quantization with explicit interest generation. The IDMM maintains a shared, discrete Interest Dictionary $\bm{E}^* = \prod_{c=1}^C \bm{E}^c$ using a quantization objective that induces a Voronoi partition, while the decoupled MIPDM employs a user-conditioned GPT to model the next-step distribution over multi-dimensional interests. The MIRM retrieves items using multiple user-interest representations that are formed by fusing the selected interest embedding $\mathbf{e}_{m^*}$ with the user embedding, trained with a joint objective $\mathcal{L}_{\text{total}} = \lambda_1 \mathcal{L}_{IDMM} + \lambda_2 \mathcal{L}_{MIPDM} + \lambda_3 \mathcal{L}_{MIRM}$. Theoretical analysis links quantization to separability via Voronoi cells, and extensive experiments across four real-world datasets—including a production deployment on Rednote—demonstrate superior performance, improved diversity across interests, and effective evolution modeling, confirming the practical value of the framework for industrial retrieval systems.

Abstract

Multi-interest recommendation has gained attention, especially in industrial retrieval stage. Unlike classical dual-tower methods, it generates multiple user representations instead of a single one to model comprehensive user interests. However, prior studies have identified two underlying limitations: the first is interest collapse, where multiple representations homogenize. The second is insufficient modeling of interest evolution, as they struggle to capture latent interests absent from a user's historical behavior. We begin with a thorough review of existing works in tackling these limitations. Then, we attempt to tackle these limitations from a new perspective. Specifically, we propose a framework-level refinement for multi-interest recommendation, named GemiRec. The proposed framework leverages interest quantization to enforce a structural interest separation and interest generation to learn the evolving dynamics of user interests explicitly. It comprises three modules: (a) Interest Dictionary Maintenance Module (IDMM) maintains a shared quantized interest dictionary. (b) Multi-Interest Posterior Distribution Module (MIPDM) employs a generative model to capture the distribution of user future interests. (c) Multi-Interest Retrieval Module (MIRM) retrieves items using multiple user-interest representations. Both theoretical and empirical analyses, as well as extensive experiments, demonstrate its advantages and effectiveness. Moreover, it has been deployed in production since March 2025, showing its practical value in industrial applications.

Figures (5)

• Figure 1: Illustration of the underlying challenges: (1). Interest collapse, where multiple interest representations primarily retrieve maternal and baby items (pink). (2). Insufficient modeling of interest evolution, where the learned interests fail to retrieve yoga-related items (purple) that are absent from the user's behavior sequence.
• Figure 2: Overview of the GemiRec, illustrating the integration and interaction between the IDMM, MIPDM, and MIRM . (a) IDMM: maintaining multiple vector-quantized sub-dictionaries containing discrete interest embeddings; (b) MIPDM: employing a decoupled generative model to capture the distribution of user interests at the next time step; (c) MIRM: retrieving items using multiple user-interest representations.
• Figure 3: Distribution and importance of Interest Dictionary.
• Figure 4: t-SNE visualization of user-interest representations, with embeddings of previously interacted and future-interacted items.
• Figure 5: Hyperparameter analysis on K and $\varepsilon$.

Theorems & Definitions (10)

• Proposition 1: Interest Quantization Induces a Voronoi Partition
• Corollary 1: Structural Separation Induced by Discrete Indices
• Corollary 2: Amplified Separation in Interest Dictionary
• Proposition 2: From Interest Dictionary separability to retrieval separability
• Proposition 3
• Definition 1: Voronoi Partition
• Proposition 4: Equivalent Characterization
• Proposition 5: Proof of Interest Quantization Induces a Voronoi Partition
• Proposition 6: Proof of From Interest Dictionary separability to retrieval separability
• Proposition 7