Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Chain of Behavior for User Trajectory Prediction

Chengkai Huang, Xiaodi Chen, Hongtao Huang, Quan Z. Sheng, Lina Yao

TL;DR

The paper tackles long-horizon user trajectory prediction by moving beyond next-item prediction to generate multi-step action sequences. It introduces Generative Chain of Behavior (GCB), which maps items into a discrete Semantic-ID space via a Residual Quantized VAE with k-means initialization and then uses a Transformer-based autoregressive generator to produce coherent future action chains over $k$ steps. Key contributions include the RQ-VAE with hierarchical codebooks, a robust semantic tokenization scheme, and a T5-style encoder–decoder with beam search for multi-step generation in the semantic space. Experiments on Amazon Beauty and Cell Phones datasets show state-of-the-art multi-step accuracy and trajectory coherence, highlighting the method's effectiveness in modeling long-horizon intent drift. The work offers a unified, generative formulation for capturing evolving user preferences with practical implications for proactive, long-term recommendations.

Abstract

Modeling long-term user behavior trajectories is essential for understanding evolving preferences and enabling proactive recommendations. However, most sequential recommenders focus on next-item prediction, overlooking dependencies across multiple future actions. We propose Generative Chain of Behavior (GCB), a generative framework that models user interactions as an autoregressive chain of semantic behaviors over multiple future steps. GCB first encodes items into semantic IDs via RQ-VAE with k-means refinement, forming a discrete latent space that preserves semantic proximity. On top of this space, a transformer-based autoregressive generator predicts multi-step future behaviors conditioned on user history, capturing long-horizon intent transitions and generating coherent trajectories. Experiments on benchmark datasets show that GCB consistently outperforms state-of-the-art sequential recommenders in multi-step accuracy and trajectory consistency. Beyond these gains, GCB offers a unified generative formulation for capturing user preference evolution.

Generative Chain of Behavior for User Trajectory Prediction

TL;DR

The paper tackles long-horizon user trajectory prediction by moving beyond next-item prediction to generate multi-step action sequences. It introduces Generative Chain of Behavior (GCB), which maps items into a discrete Semantic-ID space via a Residual Quantized VAE with k-means initialization and then uses a Transformer-based autoregressive generator to produce coherent future action chains over steps. Key contributions include the RQ-VAE with hierarchical codebooks, a robust semantic tokenization scheme, and a T5-style encoder–decoder with beam search for multi-step generation in the semantic space. Experiments on Amazon Beauty and Cell Phones datasets show state-of-the-art multi-step accuracy and trajectory coherence, highlighting the method's effectiveness in modeling long-horizon intent drift. The work offers a unified, generative formulation for capturing evolving user preferences with practical implications for proactive, long-term recommendations.

Abstract

Modeling long-term user behavior trajectories is essential for understanding evolving preferences and enabling proactive recommendations. However, most sequential recommenders focus on next-item prediction, overlooking dependencies across multiple future actions. We propose Generative Chain of Behavior (GCB), a generative framework that models user interactions as an autoregressive chain of semantic behaviors over multiple future steps. GCB first encodes items into semantic IDs via RQ-VAE with k-means refinement, forming a discrete latent space that preserves semantic proximity. On top of this space, a transformer-based autoregressive generator predicts multi-step future behaviors conditioned on user history, capturing long-horizon intent transitions and generating coherent trajectories. Experiments on benchmark datasets show that GCB consistently outperforms state-of-the-art sequential recommenders in multi-step accuracy and trajectory consistency. Beyond these gains, GCB offers a unified generative formulation for capturing user preference evolution.
Paper Structure (12 sections, 20 equations, 2 figures, 1 table)

This paper contains 12 sections, 20 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Methodology
  4. Semantic-ID Encoding with RQ-VAE
  5. Residual Quantized VAE
  6. K-means Codebook Initialization
  7. Objective and Semantic-ID Tokens
  8. Generative User Behaviour Trajectory Prediction
  9. Tokenization of User Histories and Futures
  10. Beam Search Generation in Semantic-ID Space
  11. Experiments
  12. Conclusion

Figures (2)

  • Figure 1: The overview of our GCB framework.
  • Figure 2: Overall category distribution in L1 and L2 clusters.