Unlocking Scaling Law in Industrial Recommendation Systems with a Three-step Paradigm based Large User Model

TL;DR

This work addresses the gap between the scaling laws of autoregressive models and the practical needs of industrial recommendation systems. It introduces a Large User Model (LUM) built around a three-step paradigm: knowledge construction via generative pre-training, knowledge querying under conditional prompts, and knowledge utilization to augment traditional DLRMs for retrieval and ranking. The key innovations include tokenizing UBS into separate condition and item tokens to enable next-condition-item prediction, leveraging InfoNCE loss with packing for efficiency, and a group-query mechanism to accelerate inference. Empirical results show strong offline performance on public datasets, substantial gains in industrial settings, and a successful Taobao deployment with measurable CTR and RPM improvements, demonstrating both scalability and real-world impact.

Abstract

Recent advancements in autoregressive Large Language Models (LLMs) have achieved significant milestones, largely attributed to their scalability, often referred to as the "scaling law". Inspired by these achievements, there has been a growing interest in adapting LLMs for Recommendation Systems (RecSys) by reformulating RecSys tasks into generative problems. However, these End-to-End Generative Recommendation (E2E-GR) methods tend to prioritize idealized goals, often at the expense of the practical advantages offered by traditional Deep Learning based Recommendation Models (DLRMs) in terms of in features, architecture, and practices. This disparity between idealized goals and practical needs introduces several challenges and limitations, locking the scaling law in industrial RecSys. In this paper, we introduce a large user model (LUM) that addresses these limitations through a three-step paradigm, designed to meet the stringent requirements of industrial settings while unlocking the potential for scalable recommendations. Our extensive experimental evaluations demonstrate that LUM outperforms both state-of-the-art DLRMs and E2E-GR approaches. Notably, LUM exhibits excellent scalability, with performance improvements observed as the model scales up to 7 billion parameters. Additionally, we have successfully deployed LUM in an industrial application, where it achieved significant gains in an A/B test, further validating its effectiveness and practicality.

Figures (6)

• Figure 1: Intuitive insight from the common paradigm in using LLM to the proposed multi-step, generative-to-discriminative paradigm.
• Figure 2: The comparison of different tokenizations
• Figure 3: (a) The architecture of LUM. (b) An example of query knowledge from pre-trained LUM. (c) An example of utilizing knowledge in DLRMs.
• Figure 4: An example of group query.
• Figure 5: The Results of Efficiency Evaluation.