M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework

TL;DR

M3oE is the first effort to solve multi-domain multi-task recommendation self-adaptively, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework that integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives.

Abstract

Multi-domain recommendation and multi-task recommendation have demonstrated their effectiveness in leveraging common information from different domains and objectives for comprehensive user modeling. Nonetheless, the practical recommendation usually faces multiple domains and tasks simultaneously, which cannot be well-addressed by current methods. To this end, we introduce M3oE, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. M3oE integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives. We leverage three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences respectively to address the complex dependencies among multiple domains and tasks in a disentangled manner. Additionally, we design a two-level fusion mechanism for precise control over feature extraction and fusion across diverse domains and tasks. The framework's adaptability is further enhanced by applying AutoML technique, which allows dynamic structure optimization. To the best of the authors' knowledge, our M3oE is the first effort to solve multi-domain multi-task recommendation self-adaptively. Extensive experiments on two benchmark datasets against diverse baselines demonstrate M3oE's superior performance. The implementation code is available to ensure reproducibility.

Figures (4)

• Figure 1: Multi-domain multi-task AUC comparisons on MovieLens. We report the relative improvement of MMoE, STAR, and our M3oE, compared with single-domain single-task MLP baseline. The different colors indicate different performance ranks in the same domain and task.
• Figure 2: Framework of M3oE, focusing on domain $d$ and task $t$ for clarity. Within the multi-view layer, there are three modules arranged from left to right: shared expert module $\boldsymbol{\mathcal{S}}$, domain expert module $\boldsymbol{\mathcal{D}}$, and task expert module $\boldsymbol{\mathcal{T}}$.
• Figure 3: T-SNE results on MovieLens domain 1 task 1 (left column) and KuaiRand-Pure domain 2 task 1 (right column).
• Figure 4: Hyper-parameter analysis results.