Query as Anchor: Scenario-Adaptive User Representation via Large Language Model

TL;DR

This work proposes Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis, and introduces Cluster-based Soft Prompt Tuning to enforce discriminative latent structures, effectively aligning model attention with scenario-specific modalities.

Abstract

Industrial-scale user representation learning requires balancing robust universality with acute task-sensitivity. However, existing paradigms primarily yield static, task-agnostic embeddings that struggle to reconcile the divergent requirements of downstream scenarios within unified vector spaces. Furthermore, heterogeneous multi-source data introduces inherent noise and modality conflicts, degrading representation. We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis. To empower Large Language Models (LLMs) with deep user understanding, we first construct UserU, an industrial-scale pre-training dataset that aligns multi-modal behavioral sequences with user understanding semantics, and our Q-Anchor Embedding architecture integrates hierarchical coarse-to-fine encoders into dual-tower LLMs via joint contrastive-autoregressive optimization for query-aware user representation. To bridge the gap between general pre-training and specialized business logic, we further introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures, effectively aligning model attention with scenario-specific modalities. For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency. Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment. Large-scale online A/B testing in Alipay's production system across two real-world scenarios further validates its practical effectiveness. Our code is prepared for public release and will be available at: https://github.com/JhCircle/Q-Anchor.

Figures (12)

• Figure 1: Comparison between (A) General User Embedding dou2025transferable and (B) our Query-as-anchor. (A) learns transferable user representations across domains but generates fixed embeddings regardless of downstream context. (B) extends (A) with query-as-anchor modulation, enabling a single model to produce adaptive, domain-specific embeddings via natural language instructions.
• Figure 2: Overview of Query-as-Anchor Framework for our Q-Anchor Embedding.
• Figure 3: Input Template of UserU. The Hierarchical User Tokens: $\mathbf{e}_i$ injects the precomputed hierarchical embedding $\mathbf{e}_i$ (Eq. \ref{['eq:tokens']}). An optional instruction is followed by the special <USER_EMB> token, which signals the model to extract a unified user embedding (Sec. \ref{['subsubsec:pretrain']}).
• Figure 4: KV-Cache optimized Query-as-Anchor inference: pre-computed user prefixes enable sequence, low-latency re-anchoring across diverse tasks.
• Figure 5: Average KS performance of Q-Anchor and baselines across 10 Alipay scenarios. Per-scenario results are provided in Appendix \ref{['subapp:ks']}.