MergeRec: Model Merging for Data-Isolated Cross-Domain Sequential Recommendation

TL;DR

Compared to conventional model merging methods, MergeRec consistently achieves superior performance, with average improvements of up to 17.21% in Recall@10, highlighting the potential of model merging as a scalable and effective approach for building universal recommender systems.

Abstract

Modern recommender systems trained on domain-specific data often struggle to generalize across multiple domains. Cross-domain sequential recommendation has emerged as a promising research direction to address this challenge; however, existing approaches face fundamental limitations, such as reliance on overlapping users or items across domains, or unrealistic assumptions that ignore privacy constraints. In this work, we propose a new framework, MergeRec, based on model merging under a new and realistic problem setting termed data-isolated cross-domain sequential recommendation, where raw user interaction data cannot be shared across domains. MergeRec consists of three key components: (1) merging initialization, (2) pseudo-user data construction, and (3) collaborative merging optimization. First, we initialize a merged model using training-free merging techniques. Next, we construct pseudo-user data by treating each item as a virtual sequence in each domain, enabling the synthesis of meaningful training samples without relying on real user interactions. Finally, we optimize domain-specific merging weights through a joint objective that combines a recommendation loss, which encourages the merged model to identify relevant items, and a distillation loss, which transfers collaborative filtering signals from the fine-tuned source models. Extensive experiments demonstrate that MergeRec not only preserves the strengths of the original models but also significantly enhances generalizability to unseen domains. Compared to conventional model merging methods, MergeRec consistently achieves superior performance, with average improvements of up to 17.21% in Recall@10, highlighting the potential of model merging as a scalable and effective approach for building universal recommender systems. The source code is available at https://github.com/DIALLab-SKKU/MergeRec.

Figures (8)

• Figure 1: Illustration of training and test user sequences in sequential recommendation across multiple domains. Each test user sequence (black box) contains all previous interactions, including those from the training period (blue box), highlighting that test data are a superset of training data in real-world scenarios.
• Figure 2: Overview of MergeRec with three main components. (a) Merging initialization integrates into a unified model containing multi-domain knowledge. (b) Pseudo-user data construction creates a single-item sequence. (c) Collaborative merging optimization jointly optimizes the recommendation loss $\mathcal{L}_{Rec}$ and the distillation loss $\mathcal{L}_{KD}$.
• Figure 3: Cross-entropy loss and prediction entropy dynamics of AdaMerging and MergeRec (Ours) over training steps.
• Figure 4: Normalized average performance across varying the number of datasets for model merging.
• Figure 5: Normalized average performance across user and item groups on eight datasets. User and item groups are divided by sequence length and item popularity, respectively.