Transferable and Forecastable User Targeting Foundation Model

TL;DR

FOUND tackles the dual challenges of cross-domain transferability and forecastability in user targeting by integrating heterogeneous multi-scenario data and forecastable user-text pairs through a two-stage self-supervised framework and contrastive alignment with a language model. The method combines time-aware behavioral encoding, tabular and textual encoders, and cross-attention fusion to produce rich, transferable user representations aligned with future-behavior descriptions. Zero-shot transfer and few-shot prompt-tuning enable practical deployment across domains, with extensive real-world results on Alipay benchmarks showing clear gains over strong baselines. The work demonstrates strong cross-domain generalization and practical impact, including deployment in production and notable improvements in metrics such as CTR in various applications.

Abstract

User targeting, the process of selecting targeted users from a pool of candidates for non-expert marketers, has garnered substantial attention with the advancements in digital marketing. However, existing user targeting methods encounter two significant challenges: (i) Poor cross-domain and cross-scenario transferability and generalization, and (ii) Insufficient forecastability in real-world applications. These limitations hinder their applicability across diverse industrial scenarios. In this work, we propose FOUND, an industrial-grade, transferable, and forecastable user targeting foundation model. To enhance cross-domain transferability, our framework integrates heterogeneous multi-scenario user data, aligning them with one-sentence targeting demand inputs through contrastive pre-training. For improved forecastability, the text description of each user is derived based on anticipated future behaviors, while user representations are constructed from historical information. Experimental results demonstrate that our approach significantly outperforms existing baselines in cross-domain, real-world user targeting scenarios, showcasing the superior capabilities of FOUND. Moreover, our method has been successfully deployed on the Alipay platform and is widely utilized across various scenarios.

Figures (6)

• Figure 1: Pre-training and inference for user targeting foundation model. We propose an industrial framework for language-based user targeting, i.e., selecting users from only one sentence. it' pre-trained in a two strategy, as the first stage focuses on the self-supervised user modeling while the second is employed to align user-text pairs. For user targeting, we develop two approaches: zero-shot transfer, which selects users with one-sentence input, and few-shot user targeting via prompt-tuning.
• Figure 2: Self-supervised pre-training tasks for user behavioral sequence.
• Figure 3: Few-shot User Targeting. The embeddings of both targeted, untargeted and hard-negative users are visualized with t-SNE, presented in the same coordinate system together with prompt embedding.
• Figure 4: t-SNE visualization of user representations. Every color represents users with different attributes (here denoting the outcomes of each user).
• Figure 5: The architecture of our tabular encoder.