Towards Distribution Matching between Collaborative and Language Spaces for Generative Recommendation

TL;DR

This work addresses the misalignment between collaborative and language representations in generative recommendation by introducing DMRec, a model-agnostic framework that uses a probabilistic meta-network to bridge outputs of language models with user interaction signals. It defines two probabilistic spaces, $\mathcal{X}$ (collaborative) and $\mathcal{Y}$ (language), models $q_{\phi}(\mathbf z|\mathbf x)$ and $p_{\varphi}(\mathbf z|\mathbf s)$, and employs three cross-space distribution matching strategies—GODM, CPDM, and MDDM—to align distributions while preserving space-specific semantics. The approach is validated on three public datasets across three base generative models, showing consistent improvements over both base models and LM-enhanced baselines, with particular strength in sparse-data scenarios. DMRec demonstrates training efficiency gains and robust performance, providing a practical, plug-and-play method to leverage language-model guidance in probabilistic generative recommendations.

Abstract

Generative recommendation aims to learn the underlying generative process over the entire item set to produce recommendations for users. Although it leverages non-linear probabilistic models to surpass the limited modeling capacity of linear factor models, it is often constrained by a trade-off between representation ability and tractability. With the rise of a new generation of generative methods based on pre-trained language models (LMs), incorporating LMs into general recommendation with implicit feedback has gained considerable attention. However, adapting them to generative recommendation remains challenging. The core reason lies in the mismatch between the input-output formats and semantics of generative models and LMs, making it challenging to achieve optimal alignment in the feature space. This work addresses this issue by proposing a model-agnostic generative recommendation framework called DMRec, which introduces a probabilistic meta-network to bridge the outputs of LMs with user interactions, thereby enabling an equivalent probabilistic modeling process. Subsequently, we design three cross-space distribution matching processes aimed at maximizing shared information while preserving the unique semantics of each space and filtering out irrelevant information. We apply DMRec to three different types of generative recommendation methods and conduct extensive experiments on three public datasets. The experimental results demonstrate that DMRec can effectively enhance the recommendation performance of these generative models, and it shows significant advantages over mainstream LM-enhanced recommendation methods.

Figures (6)

• Figure 1: The proposed DMRec information flow, which models user preference distributions $q_\phi$ and $p_\varphi$ in the collaborative space $\mathcal{X}$ and language space $\mathcal{Y}$, respectively, and performs cross-space distribution matching through GODM, CPDM, or MDDM.
• Figure 2: Comparison of the training process and speed of the base model and DMRecw.r.t. Recall@20 on validation sets. The red dot indicates the best-performing on test sets.
• Figure 3: Comparison of the base model and DMRec performance across sparse user groups. The bar chart (left $y$-axis) shows user count, the line chart (right $y$-axis) shows NDCG@20, and the $x$-axis displays the number of interactions and user group proportions.
• Figure 4: Comparison of the base model and DMrec for user distribution dimension activity $\text{log} a^{\phi}$ on three datasets.
• Figure 5: Ablation studies on (a) Yelp and (b) Steam datasets w.r.t. Recall@20 (left $y$-axis) and NDCG@20 (right $y$-axis).