Principled Synthetic Data Enables the First Scaling Laws for LLMs in Recommendation

TL;DR

The paper tackles the absence of predictable scaling laws for LLMs in recommendation by identifying data quality as the core bottleneck. It introduces a layered synthetic data framework that builds a two-layer curriculum: Layer 1 grounds semantics and collaborative signals via item-text alignment and CF data, and Layer 2 generates unbiased user interaction histories through graph-based random walks. Empirical results show that models trained on this principled synthetic data outperform real-data baselines in downstream ranking and exhibit robust power-law scaling across data modalities and model sizes, with clear hierarchies in pedagogical efficiency (UIH > CF > Item-Text > General). These findings provide a data-centric blueprint for reliable scaling in recommender LLMs, enabling more predictable resource planning and reducing dependence on biased real-world logs.

Abstract

Large Language Models (LLMs) represent a promising frontier for recommender systems, yet their development has been impeded by the absence of predictable scaling laws, which are crucial for guiding research and optimizing resource allocation. We hypothesize that this may be attributed to the inherent noise, bias, and incompleteness of raw user interaction data in prior continual pre-training (CPT) efforts. This paper introduces a novel, layered framework for generating high-quality synthetic data that circumvents such issues by creating a curated, pedagogical curriculum for the LLM. We provide powerful, direct evidence for the utility of our curriculum by showing that standard sequential models trained on our principled synthetic data significantly outperform ($+130\%$ on recall@100 for SasRec) models trained on real data in downstream ranking tasks, demonstrating its superiority for learning generalizable user preference patterns. Building on this, we empirically demonstrate, for the first time, robust power-law scaling for an LLM that is continually pre-trained on our high-quality, recommendation-specific data. Our experiments reveal consistent and predictable perplexity reduction across multiple synthetic data modalities. These findings establish a foundational methodology for reliable scaling LLM capabilities in the recommendation domain, thereby shifting the research focus from mitigating data deficiencies to leveraging high-quality, structured information.

Figures (12)

• Figure 1: TSTR vs TRTR: Recall@K Comparison Across Models. TSTR (Train on Synthetic, Test on Real) consistently outperforms TRTR (Train on Real, Test on Real) across all models (GRU4Rec, NARM, SASRec, STAMP) and K-values. Note: Real UIHs are filtered to the same set of items as Synthetic UIHs.
• Figure 2: Scaling laws for different domains across different model scales. UIH exhibits strong scaling ($\alpha_{\text{UIH}} = 0.63$--$0.99$), indicating continued improvement with additional training tokens. General domain shows near-saturation ($\alpha < 0.1$) as expected for pretrained models. The dashed lines are the fitted scaling law curves and parameters are provided as legend of the curve and Table \ref{['tab:model_scale']}
• Figure 3: Ablation studies on selection of domains for recommendation data. Obviously, excluding a domain will causes degradation on the corresponding domain.
• Figure 4: We plot the evaluation perplexity for UIH using number of UIH training tokens as X-axis. This figure indicates including CF data (Item-text + CF + UIH and CF + UIH) enables the model to learn UIH better.
• Figure 5: Scaling laws on 4B models with different mixture ratio on UIH. Obviously the perplexity starts to increase when the UIH mixture ratio is too high (reduced UIH data is repeated too many times), which is a sign of overfitting. The higher mixture ratio, this increase happens earlier in training stage. We omitted the figures for CF Both Seen and CF On Unseen here, as they are very similar to CF Both Unseen.