Adaptive$^2$: Adaptive Domain Mining for Fine-grained Domain Adaptation Modeling

TL;DR

This work proposes Adaptive, a novel framework that first learns domains adaptively using a domain mining module by self-supervision and then employs a shared&specific network to model shared and conflicting information in online advertisement.

Abstract

Advertising systems often face the multi-domain challenge, where data distributions vary significantly across scenarios. Existing domain adaptation methods primarily focus on building domain-adaptive neural networks but often rely on hand-crafted domain information, e.g., advertising placement, which may be sub-optimal. We think that fine-grained "domain" patterns exist that are difficult to hand-craft in online advertisement. Thus, we propose Adaptive$^2$, a novel framework that first learns domains adaptively using a domain mining module by self-supervision and then employs a shared&specific network to model shared and conflicting information. As a practice, we use VQ-VAE as the domain mining module and conduct extensive experiments on public benchmarks. Results show that traditional domain adaptation methods with hand-crafted domains perform no better than single-domain models under fair FLOPS conditions, highlighting the importance of domain definition. In contrast, Adaptive$^2$ outperforms existing approaches, emphasizing the effectiveness of our method and the significance of domain mining. We also deployed Adaptive$^2$ in the live streaming scenario of Kuaishou Advertising System, demonstrating its commercial value and potential for automatic domain identification. To the best of our knowledge, Adaptive$^2$ is the first approach to automatically learn both domain identification and adaptation in online advertising, opening new research directions for this area.

Figures (6)

• Figure 1: The key tasks of solving multi-domain problems and the corresponding components in Adaptive$^2$ are as above. Part (a) indicates the two aspects that domain adaptation should address, while part (b) shows the specific modules related to those aspects.
• Figure 2: The Adaptive$^2$ framework, consists of the domain mining and the adaptive domain modeling module. The domain mining module outputs the domain index of each sample, which is used for routing to the domain-specific networks in the adaptive domain modeling module. Specifically, we employ VQVAE as the domain mining module.
• Figure 3: Parameter comparison on two datasets. The horizontal axis represents computational FLOPs, the vertical axis represents AUC, and the numbers next to the models indicate the parameter count as a multiple of the MLP's parameters. The setting is identical to the main result.
• Figure 4: Visualization of the embeddings before VQ-VAE encoder on avazu.
• Figure 5: Visualization of the embeddings after VQ-VAE encoder on avazu.