Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MLoRA: Multi-Domain Low-Rank Adaptive Network for CTR Prediction

Zhiming Yang, Haining Gao, Dehong Gao, Luwei Yang, Libin Yang, Xiaoyan Cai, Wei Ning, Guannan Zhang

TL;DR

MLoRA introduces a parameter-efficient, domain-specific adaptation mechanism for multi-domain CTR prediction by attaching per-domain Low-Rank Adaptor (LoRA) modules to a shared backbone. The method splits each layer into a common part and a domain-specific low-rank component, enabling efficient learning of domain-specific patterns while preserving shared representations. Extensive offline experiments across Taobao, Amazon, and Movielens datasets show consistent WAUC gains over strong baselines, and a real-world Alibaba deployment demonstrates meaningful online improvements with only a small parameter overhead. The work contributes a model-agnostic, scalable framework that can readily be applied to diverse CTR architectures and new domains, validated by substantial production gains and publicly released code.

Abstract

Click-through rate (CTR) prediction is one of the fundamental tasks in the industry, especially in e-commerce, social media, and streaming media. It directly impacts website revenues, user satisfaction, and user retention. However, real-world production platforms often encompass various domains to cater for diverse customer needs. Traditional CTR prediction models struggle in multi-domain recommendation scenarios, facing challenges of data sparsity and disparate data distributions across domains. Existing multi-domain recommendation approaches introduce specific-domain modules for each domain, which partially address these issues but often significantly increase model parameters and lead to insufficient training. In this paper, we propose a Multi-domain Low-Rank Adaptive network (MLoRA) for CTR prediction, where we introduce a specialized LoRA module for each domain. This approach enhances the model's performance in multi-domain CTR prediction tasks and is able to be applied to various deep-learning models. We evaluate the proposed method on several multi-domain datasets. Experimental results demonstrate our MLoRA approach achieves a significant improvement compared with state-of-the-art baselines. Furthermore, we deploy it in the production environment of the Alibaba.COM. The online A/B testing results indicate the superiority and flexibility in real-world production environments. The code of our MLoRA is publicly available.

MLoRA: Multi-Domain Low-Rank Adaptive Network for CTR Prediction

TL;DR

MLoRA introduces a parameter-efficient, domain-specific adaptation mechanism for multi-domain CTR prediction by attaching per-domain Low-Rank Adaptor (LoRA) modules to a shared backbone. The method splits each layer into a common part and a domain-specific low-rank component, enabling efficient learning of domain-specific patterns while preserving shared representations. Extensive offline experiments across Taobao, Amazon, and Movielens datasets show consistent WAUC gains over strong baselines, and a real-world Alibaba deployment demonstrates meaningful online improvements with only a small parameter overhead. The work contributes a model-agnostic, scalable framework that can readily be applied to diverse CTR architectures and new domains, validated by substantial production gains and publicly released code.

Abstract

Click-through rate (CTR) prediction is one of the fundamental tasks in the industry, especially in e-commerce, social media, and streaming media. It directly impacts website revenues, user satisfaction, and user retention. However, real-world production platforms often encompass various domains to cater for diverse customer needs. Traditional CTR prediction models struggle in multi-domain recommendation scenarios, facing challenges of data sparsity and disparate data distributions across domains. Existing multi-domain recommendation approaches introduce specific-domain modules for each domain, which partially address these issues but often significantly increase model parameters and lead to insufficient training. In this paper, we propose a Multi-domain Low-Rank Adaptive network (MLoRA) for CTR prediction, where we introduce a specialized LoRA module for each domain. This approach enhances the model's performance in multi-domain CTR prediction tasks and is able to be applied to various deep-learning models. We evaluate the proposed method on several multi-domain datasets. Experimental results demonstrate our MLoRA approach achieves a significant improvement compared with state-of-the-art baselines. Furthermore, we deploy it in the production environment of the Alibaba.COM. The online A/B testing results indicate the superiority and flexibility in real-world production environments. The code of our MLoRA is publicly available.
Paper Structure (16 sections, 12 equations, 4 figures, 6 tables)

This paper contains 16 sections, 12 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. CTR Prediction
  4. Low-Rank Adaptor
  5. Methodology
  6. Problem definition
  7. Low-Rank Adaptor
  8. MLoRA for Multi-domain CTR
  9. Experiments
  10. Datasets
  11. Baselines
  12. Settings
  13. Main Experiments
  14. Ablation Studies
  15. Industry Application
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Upper: three representative business domains of Alibaba.com, Get product inspiration, Recommend in miniDetail and Recommend from this supplier. Lower: three key business sectors of Amazon, Banner, Selected for you and Recommend after card.
  • Figure 2: (a) Training separated model for each domain. (b) Training a unified model by mixing multi-domain data. (c) Multi-domain Framework STAR_CIKMMMoE_SIGKDDPLE_recsys. (d) Our Proposed MLoRA.
  • Figure 3: The overview architecture of the proposed MLoRA, which consists of input features, embedding layer, shared network, multi-domain network, and domain selector. Each domain network consists of low-rank matrix $\mathbf{A}$ and $\mathbf{B}$, and the outputs of the domain network and share network are added together to serve as the output of the entire layer. The embedding layer and shared network are trained during the pretraining phase, while the domain network is trained during the finetuning phase.
  • Figure 4: As temperature coefficient $\alpha$ changes, the performance of MLP, FiBiNET, and DeepFM with MLoRA varies on the Taobao-10 dataset.