PAP-REC: Personalized Automatic Prompt for Recommendation Language Model

TL;DR

This work addresses the labor-intensive process of crafting prompts for recommendation language models by introducing PAP-REC, an automatic framework that generates personalized prompts via a gradient-based trigger-token search augmented with surrogate evaluation metrics. It tackles the enormous search space through an iterative, per-user token update scheme and uses surrogate metrics to efficiently optimize top-k recommendation performance metrics. Across three real-world datasets and three recommendation tasks, PAP-REC-generated prompts consistently outperform manually designed prompts and baselines, with notable gains from personalization in several settings. The approach enables scalable, plug-and-play enhancement of RLM-driven recommendations and opens avenues for integrating item-ID strategies or exploring reinforcement learning or large-language-model-based prompt generation in future work.

Abstract

Recently emerged prompt-based Recommendation Language Models (RLM) can solve multiple recommendation tasks uniformly. The RLMs make full use of the inherited knowledge learned from the abundant pre-training data to solve the downstream recommendation tasks by prompts, without introducing additional parameters or network training. However, handcrafted prompts require significant expertise and human effort since slightly rewriting prompts may cause massive performance changes. In this paper, we propose PAP-REC, a framework to generate the Personalized Automatic Prompt for RECommendation language models to mitigate the inefficiency and ineffectiveness problems derived from manually designed prompts. Specifically, personalized automatic prompts allow different users to have different prompt tokens for the same task, automatically generated using a gradient-based method. One challenge for personalized automatic prompt generation for recommendation language models is the extremely large search space, leading to a long convergence time. To effectively and efficiently address the problem, we develop surrogate metrics and leverage an alternative updating schedule for prompting recommendation language models. Experimental results show that our PAP-REC framework manages to generate personalized prompts, and the automatically generated prompts outperform manually constructed prompts and also outperform various baseline recommendation models. The source code of the work is available at https://github.com/rutgerswiselab/PAP-REC.

Figures (4)

• Figure 1: Overview of the iterative automated personalized and non-personalized prompt generation process.
• Figure 2: An illustration of the gradient-based prompt search method in PAP-REC. The example is an automated personalized prompt with {task arg} followed by the task-specific token as "Choose describes consideration recommend Mid" and a specific token to the $u_{47}$, "nominee". PAP-REC will apply the generated prompt to the training data and record the gradient of trigger token embeddings. For non-personalized prompt search, the last token embedding $e_u$ will be $e_{r_6}$.
• Figure 3: Sequential recommendation performance under different prompt lengths.
• Figure 4: Ablation study on surrogate metrics $\widetilde{M}$ compared with loss as the selection criterion.