Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Layout Is the Model: On Action-Item Coupling in Generative Recommendation

Xiaokai Wei, Jiajun Wu, Daiyao Yi, Reza Shirkavand, Michelle Gong

TL;DR

This work addresses how token layout in generative recommender systems impacts information use, leakage safety, and efficiency. It formalizes a first-principles approach to layout design and introduces Lagged Action Conditioning (LAC), a non-interleaved layout that preserves the action-given-item causal direction while keeping sequence length linear. Through extensive experiments on public and industrial data, LAC achieves competitive or superior accuracy compared to interleaved and patched-conditioning baselines, while delivering substantial FLOPs reductions and faster training. The findings offer actionable guidance for building scalable GR systems that jointly optimize ranking quality and computational efficiency. Overall, the paper demonstrates that careful layout design can unlock strong performance without the overhead of interleaved architectures.

Abstract

Generative Recommendation (GR) models treat a user's interaction history as a sequence to be autoregressively predicted. When both items and actions (e.g., watch time, purchase, comment) are modeled, the layout-the ordering and visibility of item/action tokens-critically determines what information the model can use and how it generalizes. We present a unified study of token layouts for GR grounded in first principles: (P1) maximize item/action signal in both input/output space, (P2) preserve the conditioning relationship "action given item" and (P3) no information leakage. While interleaved layout (where item and action occupy separate tokens) naturally satisfies these principles, it also bloats sequence length with larger training/inference cost. On the non-interleaved front, we design a novel and effective approach, Lagged Action Conditioning (LAC), which appears strange on the surface but aligns well with the design principles to yield strong accuracy. Comprehensive experiments on public datasets and large-scale production logs evaluate different layout options and empirically verifies the design principles. Our proposed non-interleaved method, LAC, achieves competitive or superior quality at substantially lower FLOPs than interleaving. Our findings offer actionable guidance for assembling GR systems that are both accurate and efficient.

The Layout Is the Model: On Action-Item Coupling in Generative Recommendation

TL;DR

This work addresses how token layout in generative recommender systems impacts information use, leakage safety, and efficiency. It formalizes a first-principles approach to layout design and introduces Lagged Action Conditioning (LAC), a non-interleaved layout that preserves the action-given-item causal direction while keeping sequence length linear. Through extensive experiments on public and industrial data, LAC achieves competitive or superior accuracy compared to interleaved and patched-conditioning baselines, while delivering substantial FLOPs reductions and faster training. The findings offer actionable guidance for building scalable GR systems that jointly optimize ranking quality and computational efficiency. Overall, the paper demonstrates that careful layout design can unlock strong performance without the overhead of interleaved architectures.

Abstract

Generative Recommendation (GR) models treat a user's interaction history as a sequence to be autoregressively predicted. When both items and actions (e.g., watch time, purchase, comment) are modeled, the layout-the ordering and visibility of item/action tokens-critically determines what information the model can use and how it generalizes. We present a unified study of token layouts for GR grounded in first principles: (P1) maximize item/action signal in both input/output space, (P2) preserve the conditioning relationship "action given item" and (P3) no information leakage. While interleaved layout (where item and action occupy separate tokens) naturally satisfies these principles, it also bloats sequence length with larger training/inference cost. On the non-interleaved front, we design a novel and effective approach, Lagged Action Conditioning (LAC), which appears strange on the surface but aligns well with the design principles to yield strong accuracy. Comprehensive experiments on public datasets and large-scale production logs evaluate different layout options and empirically verifies the design principles. Our proposed non-interleaved method, LAC, achieves competitive or superior quality at substantially lower FLOPs than interleaving. Our findings offer actionable guidance for assembling GR systems that are both accurate and efficient.
Paper Structure (35 sections, 25 equations, 4 figures, 5 tables)

This paper contains 35 sections, 25 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Formulation
  5. Retrieval and scoring in the AR view.
  6. Retrieval in generative recommendation.
  7. Scoring/Ranking in generative recommendation.
  8. Item-only next-item training (retrieval only).
  9. Interleaved item-action: joint prediction for retrieval and ranking.
  10. Methodology: Layout Design for GR
  11. First principles for layout design
  12. Patched Conditioning
  13. Proposed: Lagged Action Conditioning
  14. Discussion
  15. Why LAC works despite an apparent input mismatch
  16. ...and 20 more sections

Figures (4)

  • Figure 1: Item-action layout in GR. a-d are examples that violate the design pricinples
  • Figure 2: Attention map illustration of the LAC model. We construct a dummy user sequence and visualize last layers attention map. Same with traditional attention map visualizations, X axis presents key index and Y axis presents query index.
  • Figure 3: Different inference scoring patterns for LAC and PC. Both approaches allow for parallel scoring of potentially many candidates
  • Figure 4: Performance Scaling Curve under different parameter scales. Our proposed LAC outperforms other variants and scales well as we increase model parameters.