WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging

TL;DR

WeaveRec tackles cross-domain sequential recommendation without requiring overlapping users/items by training a target-domain LoRA and multiple hybrid LoRAs on mixed-domain data, then merging them into a single adapter via weights $\lambda_i$ to guide target-domain inference. The key idea is to replace poorly performing source-domain components with hybrids trained on mixed data, mitigating negative transfer while preserving the efficiency and flexibility of model merging. Theoretical analysis shows the target-domain generalization bound is reduced under the WeaveRec mixing, and extensive experiments across single-source, multi-source, and cross-platform CDSR tasks demonstrate robust, state-of-the-art improvements over baselines. The approach is plug-and-play, scalable to additional domains, and maintains constant inference cost, making it practical for real-world multi-domain recommendation systems.

Abstract

Cross-Domain Sequential Recommendation (CDSR) seeks to improve user preference modeling by transferring knowledge from multiple domains. Despite the progress made in CDSR, most existing methods rely on overlapping users or items to establish cross-domain correlations-a requirement that rarely holds in real-world settings. The advent of large language models (LLM) and model-merging techniques appears to overcome this limitation by unifying multi-domain data without explicit overlaps. Yet, our empirical study shows that naively training an LLM on combined domains-or simply merging several domain-specific LLMs-often degrades performance relative to a model trained solely on the target domain. To address these challenges, we first experimentally investigate the cause of suboptimal performance in LLM-based cross-domain recommendation and model merging. Building on these insights, we introduce WeaveRec, which cross-trains multiple LoRA modules with source and target domain data in a weaving fashion, and fuses them via model merging. WeaveRec can be extended to multi-source domain scenarios and notably does not introduce additional inference-time cost in terms of latency or memory. Furthermore, we provide a theoretical guarantee that WeaveRec can reduce the upper bound of the expected error in the target domain. Extensive experiments on single-source, multi-source, and cross-platform cross-domain recommendation scenarios validate that WeaveRec effectively mitigates performance degradation and consistently outperforms baseline approaches in real-world recommendation tasks.

Figures (10)

• Figure 1: Illustration of Data Merging and Model Merging.
• Figure 2: Illustration of performance degradation under data merging and model merging. Experiments are based on four source domains: Amazon Beauty, Sports, Clothing, and Food, and two target domains: Clothing and Beauty.
• Figure 3: Performance comparison of different model merging methods when Sports is the target domain.
• Figure 4: An example prompt of WeaveRec.
• Figure 5: Illustration of our proposed WeaveRec Framework.