Federated Mixture-of-Expert for Non-Overlapped Cross-Domain Sequential Recommendation

TL;DR

FMoE-CDSR addresses non-overlapped cross-domain sequential recommendation under privacy constraints by transferring knowledge through domain-specific and global expert checkpoints within a federated learning framework. Each domain trains a local Expert while exploiting frozen global experts from other domains, with a Mixture-of-Experts gate to adaptively fuse predictions. The model employs a flexible embedding layer, a GNN-augmented and causal self-attention-based local encoder, and a gradient-isolated global adaptation pipeline, augmented by contrastive learning and next-item prediction objectives. Experiments on three real-world CDSR scenarios demonstrate substantial gains over FedAvg-based baselines, with ablations validating the necessity of local/global experts, gate routing, and multi-domain knowledge transfer, underscoring the practical value of privacy-preserving cross-domain knowledge sharing.

Abstract

In the real world, users always have multiple interests while surfing different services to enrich their daily lives, e.g., watching hot short videos/live streamings. To describe user interests precisely for a better user experience, the recent literature proposes cross-domain techniques by transferring the other related services (a.k.a. domain) knowledge to enhance the accuracy of target service prediction. In practice, naive cross-domain techniques typically require there exist some overlapped users, and sharing overall information across domains, including user historical logs, user/item embeddings, and model parameter checkpoints. Nevertheless, other domain's user-side historical logs and embeddings are not always available in real-world RecSys designing, since users may be totally non-overlapped across domains, or the privacy-preserving policy limits the personalized information sharing across domains. Thereby, a challenging but valuable problem is raised: How to empower target domain prediction accuracy by utilizing the other domain model parameters checkpoints only? To answer the question, we propose the FMoE-CDSR, which explores the non-overlapped cross-domain sequential recommendation scenario from the federated learning perspective.

Figures (6)

• Figure 1: Model steps of FedAvg and FMoE-CDSR.
• Figure 2: The local domain expert training process of domain $k$.
• Figure 3: The Information Fusion Process of MoE.
• Figure 4: FMoE-CDSR
• Figure 5: The predictive results of experts in MBG scenario.