Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Behavior Generative Recommendation

Zihan Liu, Yupeng Hou, Julian McAuley

TL;DR

MBGen reformulates multi-behavior sequential recommendation as a unified generative task that first predicts the next behavior and then the next item, using a tokenized sequence of interleaved behavior and item tokens. It introduces balanced item tokenizers (SID and CID) to balance the solution space and a position-routed Transformer with behavior-aware injections to model heterogeneous token sequences efficiently. The model supports three related tasks—target-behavior item prediction, behavior-specific item prediction, and behavior-item prediction—within a single autoregressive framework, achieving substantial improvements over state-of-the-art baselines on retail-like datasets. The approach advances MBSR by enabling explicit behavior anticipation, token-level pattern modeling, and scalable, conditional generation suitable for large-scale recommendation scenarios.

Abstract

Multi-behavior sequential recommendation (MBSR) aims to incorporate behavior types of interactions for better recommendations. Existing approaches focus on the next-item prediction objective, neglecting the value of integrating the target behavior type into the learning objective. In this paper, we propose MBGen, a novel Multi-Behavior sequential Generative recommendation framework. We formulate the MBSR task into a consecutive two-step process: (1) given item sequences, MBGen first predicts the next behavior type to frame the user intention, (2) given item sequences and a target behavior type, MBGen then predicts the next items. To model such a two-step process, we tokenize both behaviors and items into tokens and construct one single token sequence with both behaviors and items placed interleaved. Furthermore, MBGen learns to autoregressively generate the next behavior and item tokens in a unified generative recommendation paradigm, naturally enabling a multi-task capability. Additionally, we exploit the heterogeneous nature of token sequences in the generative recommendation and propose a position-routed sparse architecture to efficiently and effectively scale up models. Extensive experiments on public datasets demonstrate that MBGen significantly outperforms existing MBSR models across multiple tasks.

Multi-Behavior Generative Recommendation

TL;DR

MBGen reformulates multi-behavior sequential recommendation as a unified generative task that first predicts the next behavior and then the next item, using a tokenized sequence of interleaved behavior and item tokens. It introduces balanced item tokenizers (SID and CID) to balance the solution space and a position-routed Transformer with behavior-aware injections to model heterogeneous token sequences efficiently. The model supports three related tasks—target-behavior item prediction, behavior-specific item prediction, and behavior-item prediction—within a single autoregressive framework, achieving substantial improvements over state-of-the-art baselines on retail-like datasets. The approach advances MBSR by enabling explicit behavior anticipation, token-level pattern modeling, and scalable, conditional generation suitable for large-scale recommendation scenarios.

Abstract

Multi-behavior sequential recommendation (MBSR) aims to incorporate behavior types of interactions for better recommendations. Existing approaches focus on the next-item prediction objective, neglecting the value of integrating the target behavior type into the learning objective. In this paper, we propose MBGen, a novel Multi-Behavior sequential Generative recommendation framework. We formulate the MBSR task into a consecutive two-step process: (1) given item sequences, MBGen first predicts the next behavior type to frame the user intention, (2) given item sequences and a target behavior type, MBGen then predicts the next items. To model such a two-step process, we tokenize both behaviors and items into tokens and construct one single token sequence with both behaviors and items placed interleaved. Furthermore, MBGen learns to autoregressively generate the next behavior and item tokens in a unified generative recommendation paradigm, naturally enabling a multi-task capability. Additionally, we exploit the heterogeneous nature of token sequences in the generative recommendation and propose a position-routed sparse architecture to efficiently and effectively scale up models. Extensive experiments on public datasets demonstrate that MBGen significantly outperforms existing MBSR models across multiple tasks.
Paper Structure (25 sections, 1 equation, 4 figures, 7 tables)

This paper contains 25 sections, 1 equation, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Problem Formulation
  5. Learning Balanced Behavior&Item IDs
  6. Generative Multi-Task Multi-Behavior Modeling
  7. The Position-and-Behavior-Aware Transformer
  8. Experiment
  9. Experiment Setup
  10. Datasets
  11. Baselines
  12. Evaluation Settings
  13. Implementation Details
  14. Overall Performance
  15. Target Behavior Item Prediction
  16. ...and 10 more sections

Figures (4)

  • Figure 1: The overall framework of the proposed multi-behavior generative recommendation framework MBGen. We use "ut" to denote the user tokens, "bt" to denote the behavior tokens, and "it" to denote the item tokens.
  • Figure 2: Baseline models scalability comparison on target behavior item prediction task on IJCAI dataset. The model computational budget is measured from one single forward call on inference.
  • Figure 3: MBGen variants scalability comparison on behavior-specific item prediction task on IJCAI dataset. The model computational budget is measured from one single forward call on inference.
  • Figure 4: Comparison of retrieval performance with respect to different numbers of beams used in beam search along with the performance of proposed behavior aware sampling method on IJCAI (left) and Retail (right) datasets.