Learnable Item Tokenization for Generative Recommendation

TL;DR

This paper tackles the challenge of tokenizing items for LLM-based generative recommendation by introducing LETTER, a learnable tokenizer that encodes hierarchical semantics, collaborative signals, and code assignment diversity into item identifiers. LETTER leverages Residual Quantized VAE for semantic regularization, a contrastive alignment loss with collaborative CF embeddings, and a diversity loss to mitigate code bias, combined in the objective $\mathcal{L}_{LETTER}=\mathcal{L}_{Sem}+\alpha\mathcal{L}_{CF}+\beta\mathcal{L}_{Div}$. It is instantiated on two backbones (TIGER and LC-Rec) and augmented with a ranking-guided generation loss, achieving state-of-the-art results on three real-world datasets and demonstrating improved ranking performance, reduced code bias, and better alignment with collaborative signals. The work provides a practical pathway to improved open-ended recommendation with LLMs and offers insights into tokenization, regularization, and evaluation of generative recommender systems, with code and data released for reproducibility.

Abstract

Utilizing powerful Large Language Models (LLMs) for generative recommendation has attracted much attention. Nevertheless, a crucial challenge is transforming recommendation data into the language space of LLMs through effective item tokenization. Current approaches, such as ID, textual, and codebook-based identifiers, exhibit shortcomings in encoding semantic information, incorporating collaborative signals, or handling code assignment bias. To address these limitations, we propose LETTER (a LEarnable Tokenizer for generaTivE Recommendation), which integrates hierarchical semantics, collaborative signals, and code assignment diversity to satisfy the essential requirements of identifiers. LETTER incorporates Residual Quantized VAE for semantic regularization, a contrastive alignment loss for collaborative regularization, and a diversity loss to mitigate code assignment bias. We instantiate LETTER on two models and propose a ranking-guided generation loss to augment their ranking ability theoretically. Experiments on three datasets validate the superiority of LETTER, advancing the state-of-the-art in the field of LLM-based generative recommendation.

Figures (8)

• Figure 1: Overview of utilizing LETTER for LLM-based generative recommendation.
• Figure 2: Misalignment between item identifiers and collaborative signals. "Emb." denotes "Embeddings".
• Figure 3: Illustration of code assignment bias and generation bias on Instruments.
• Figure 4: Illustration of LETTER with three kinds of regularization, where semantic regularization ensures the semantic encoding, collaborative regularization enhances the alignment between the identifiers' code sequence and collaborative signals, and diversity regularization alleviates the code assignment bias.
• Figure 5: Illustration of the code assignment with biased distribution and uniform distribution of code embeddings.

Theorems & Definitions (3)

• Proposition 1
• Lemma 1
• Theorem 1