Towards Personalized Federated Multi-Scenario Multi-Task Recommendation

TL;DR

This work tackles privacy-preserving personalization in federated learning for complex recommender systems spanning multiple scenarios and tasks. It introduces PF-MSMTrec, which assigns each scenario to a dedicated client and employs a parameter template to decouple expert networks into global shared, task-specific, and scenario-specific components, enabling federated aggregation on the latter while preserving personalization. The framework integrates FedBN, conflict coordination, and personalized aggregation for experts and towers, coupled with a BCE objective and a personalization regularizer to maintain alignment between local and global parameters. Empirical results on AliExpress and Tenrec show that PF-MSMTrec outperforms state-of-the-art centralized and federated baselines, validating its effectiveness in mitigating optimization conflicts and enabling scalable, privacy-aware, multi-task recommendations across diverse scenarios.

Abstract

In modern recommender systems, especially in e-commerce, predicting multiple targets such as click-through rate (CTR) and post-view conversion rate (CTCVR) is common. Multi-task recommender systems are increasingly popular in both research and practice, as they leverage shared knowledge across diverse business scenarios to enhance performance. However, emerging real-world scenarios and data privacy concerns complicate the development of a unified multi-task recommendation model. In this paper, we propose PF-MSMTrec, a novel framework for personalized federated multi-scenario multi-task recommendation. In this framework, each scenario is assigned to a dedicated client utilizing the Multi-gate Mixture-of-Experts (MMoE) structure. To address the unique challenges of multiple optimization conflicts, we introduce a bottom-up joint learning mechanism. First, we design a parameter template to decouple the expert network parameters, distinguishing scenario-specific parameters as shared knowledge for federated parameter aggregation. Second, we implement personalized federated learning for each expert network during a federated communication round, using three modules: federated batch normalization, conflict coordination, and personalized aggregation. Finally, we conduct an additional round of personalized federated parameter aggregation on the task tower network to obtain prediction results for multiple tasks. Extensive experiments on two public datasets demonstrate that our proposed method outperforms state-of-the-art approaches. The source code and datasets will be released as open-source for public access.

Figures (7)

• Figure 1: Illustration of non-federated (a) vs. federated multi-scenario multi-task recommendation (b). In the federated setting, diverse data distributions across scenarios (clients) cause data-level conflicts. Data and model parameters stay private on each client, while the server aggregates parameters, leading to model-level conflicts. Each client also handles multiple tasks, introducing task-level conflicts.
• Figure 2: Framework of PF-MSMTrec for personalized federated multi-scenario multi-task recommendation. Each client handles a unique scenario with private data, using the MMoE structure. Parameter decoupling in expert networks enables federated aggregation of scenario-specific features. Federated batch normalization, conflict coordination, and personalized aggregation are applied in each communication round to address optimization conflicts. For clarity, Expert 3 and Expert 4 are used for Client 2, with $\hat{\mathbf{x}}_k$ and $\hat{\mathbf{x}}_{k+1}$ representing different inputs.
• Figure : (a) Ours on AliExpress.
• Figure : (a) Ours on AliExpress.
• Figure : (b) FedAVG on AliExpress.