Adaptive Conditional Expert Selection Network for Multi-domain Recommendation

TL;DR

A novel method named CESAA consists of Conditional Expert Selection (CES) Module and Adaptive Expert Aggregation (AEA) Module to tackle scalability issue and low-discriminability between domains and experts.

Abstract

Mixture-of-Experts (MOE) has recently become the de facto standard in Multi-domain recommendation (MDR) due to its powerful expressive ability. However, such MOE-based method typically employs all experts for each instance, leading to scalability issue and low-discriminability between domains and experts. Furthermore, the design of commonly used domain-specific networks exacerbates the scalability issues. To tackle the problems, We propose a novel method named CESAA consists of Conditional Expert Selection (CES) Module and Adaptive Expert Aggregation (AEA) Module to tackle these challenges. Specifically, CES first combines a sparse gating strategy with domain-shared experts. Then AEA utilizes mutual information loss to strengthen the correlations between experts and specific domains, and significantly improve the distinction between experts. As a result, only domain-shared experts and selected domain-specific experts are activated for each instance, striking a balance between computational efficiency and model performance. Experimental results on both public ranking and industrial retrieval datasets verify the effectiveness of our method in MDR tasks.

Figures (2)

• Figure 1: Overview of CESAA model architecture. CES consists of a sparse MoE and a shared expert. AEA utilizes a mutual information constraint loss to strengthen the dependency between experts and domains.
• Figure 2: Comparison of different selected sparse expert number $K$ of CESAA in Ali-CCP dataset.