Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)

TL;DR

This paper introduces P5, a unified Pretrain, Personalized Prompt & Predict Paradigm that reformulates diverse recommendation tasks as instruction-based text generation within a single encoder–decoder Transformer. By constructing a large collection of personalized prompts across five task families and pretraining on multitask input–target sequences, P5 achieves competitive or superior performance across rating, sequential, explanation, review, and direct recommendation tasks, while exhibiting zero-shot generalization to unseen prompts and new domains. The authors demonstrate the framework's robustness through extensive experiments, ablations on model size, task scaling, and prompt scaling, and provide insights into practical personalization strategies. The work positions P5 as a scalable path toward universal, instruction-based recommender systems capable of leveraging language semantics and cross-task knowledge transfer.

Abstract

For a long time, different recommendation tasks typically require designing task-specific architectures and training objectives. As a result, it is hard to transfer the learned knowledge and representations from one task to another, thus restricting the generalization ability of existing recommendation approaches, e.g., a sequential recommendation model can hardly be applied or transferred to a review generation method. To deal with such issues, considering that language can describe almost anything and language grounding is a powerful medium to represent various problems or tasks, we present a flexible and unified text-to-text paradigm called "Pretrain, Personalized Prompt, and Predict Paradigm" (P5) for recommendation, which unifies various recommendation tasks in a shared framework. In P5, all data such as user-item interactions, user descriptions, item metadata, and user reviews are converted to a common format -- natural language sequences. The rich information from natural language assists P5 to capture deeper semantics for personalization and recommendation. Specifically, P5 learns different tasks with the same language modeling objective during pretraining. Thus, it serves as the foundation model for various downstream recommendation tasks, allows easy integration with other modalities, and enables instruction-based recommendation based on prompts. P5 advances recommender systems from shallow model to deep model to big model, and will revolutionize the technical form of recommender systems towards universal recommendation engine. With adaptive personalized prompt for different users, P5 is able to make predictions in a zero-shot or few-shot manner and largely reduces the necessity for extensive fine-tuning. On several recommendation benchmarks, we conduct experiments to show the effectiveness of P5. We release the source code at https://github.com/jeykigung/P5.

Figures (6)

• Figure 1: P5 pretrains on an encoder--decoder Transformer model that takes in textual inputs and produces target responses. We trained P5 on a multitask collection of personalized prompts. After multitask prompt-based pretraining on recommendation datasets, P5 achieves the capability of zero-shot generalization to unseen personalized prompts and new items.
• Figure 2: Building input--target pairs from raw data according to our designed personalized prompt templates -- simply substituting the fields in the prompts with the corresponding information in raw data. The raw data for the five task families of P5 are from three separate sources. Specifically, rating/review/explanation prompts (a) have shared raw data. Sequential recommendation (b) and direct recommendation (c) uses similar raw data, but the former particularly requires the user interaction history. The complete collection of P5 personalized prompts are provided in the Appendix.
• Figure 3: An illustration of the P5 architecture. For the example prompt input "What star rating do you think user_23 will give item_7391?", P5 adopts an encoder--decoder framework: first encodes the input with a bidirectional text encoder, and then generates the answer through a text decoder autoregressively. In contrast to task-specific recommendation models, our P5 relies on multitask prompt-based pretraining on a large-scale personalized prompt collection, which makes P5 able to adapt to different task families and even generalize to novel personalized prompts.
• Figure 4: Example cases of zero-shot domain transfer on Z-6 task. We demonstrate three transfer directions: Toys to Beauty, Beauty to Sports, and Sports to Toys.
• Figure 5: Performance comparison among P5-S, P5-SN, and P5-PS on Beauty.