A Unified Framework for Multi-Domain CTR Prediction via Large Language Models

TL;DR

This work addresses multi-domain CTR prediction by integrating a large language model (LLM) as a backbone to capture cross-domain semantic commonalities, supplemented by domain-specific networks (DSNs) that model per-domain traits. A novel masked loss decouples DSN updates from the LLM backbone, enabling plug-and-play scalability when adding or removing domains, while a general network supports zero-shot predictions for unseen domains. The framework Uni-CTR demonstrates state-of-the-art performance on public data and strong industrial results, with efficient inference via model exporting and quantization. Overall, Uni-CTR offers a scalable, semantically rich, and generalizable approach to MDCTR that mitigates the seesaw problem and supports rapid deployment in dynamic domain settings.

Abstract

Click-Through Rate (CTR) prediction is a crucial task in online recommendation platforms as it involves estimating the probability of user engagement with advertisements or items by clicking on them. Given the availability of various services like online shopping, ride-sharing, food delivery, and professional services on commercial platforms, recommendation systems in these platforms are required to make CTR predictions across multiple domains rather than just a single domain. However, multi-domain click-through rate (MDCTR) prediction remains a challenging task in online recommendation due to the complex mutual influence between domains. Traditional MDCTR models typically encode domains as discrete identifiers, ignoring rich semantic information underlying. Consequently, they can hardly generalize to new domains. Besides, existing models can be easily dominated by some specific domains, which results in significant performance drops in the other domains (i.e. the "seesaw phenomenon"). In this paper, we propose a novel solution Uni-CTR to address the above challenges. Uni-CTR leverages a backbone Large Language Model (LLM) to learn layer-wise semantic representations that capture commonalities between domains. Uni-CTR also uses several domain-specific networks to capture the characteristics of each domain. Note that we design a masked loss strategy so that these domain-specific networks are decoupled from backbone LLM. This allows domain-specific networks to remain unchanged when incorporating new or removing domains, thereby enhancing the flexibility and scalability of the system significantly. Experimental results on three public datasets show that Uni-CTR outperforms the state-of-the-art (SOTA) MDCTR models significantly. Furthermore, Uni-CTR demonstrates remarkable effectiveness in zero-shot prediction. We have applied Uni-CTR in industrial scenarios, confirming its efficiency.

Figures (6)

• Figure 1: Prompt Template Design to consolidate domain, user, and product features for Uni-CTR.
• Figure 2: The architecture of the Uni-CTR, which takes the prompts as the input and obtains semantic representations using an LLM. Among them, domain-specific networks address the characteristics of each domain, while an additional general network aims to extract the commonalities among domains.
• Figure 3: Comparative performance of zero-shot prediction on traditional models and Uni-CTR on the unseen domain (All Beauty).
• Figure 4: Performance comparison of different language model backbones.
• Figure 5: Visualization of the representations of different layers of the LLM.