Learning to Rewrite Prompts for Personalized Text Generation

TL;DR

This work tackles personalized text generation when the underlying LLM is frozen and only accessible via API prompts. It introduces a prompt rewriter that automatically revises two context-dependent components—the summary and synthesis of personal context—within the FtPersLlm prompt, using a SL stage to constrain the RL search space and an RL stage to optimize end-to-end performance, with writing style included as a synthesis signal. Across three domains (emails, product reviews, and social media), rewritten prompts outperform the original and single-mode PROMPT optimization approaches, and the learned prompts reveal human-readable, actionable rules for manual prompt improvement. The approach enables effective personalization without fine-tuning, offering a cost-aware path for deploying personalized text generation in practical, API-restricted settings.

Abstract

Facilitated by large language models (LLMs), personalized text generation has become a rapidly growing research direction. Most existing studies focus on designing specialized models for a particular domain, or they require fine-tuning the LLMs to generate personalized text. We consider a typical scenario in which the large language model, which generates personalized output, is frozen and can only be accessed through APIs. Under this constraint, all one can do is to improve the input text (i.e., text prompts) sent to the LLM, a procedure that is usually done manually. In this paper, we propose a novel method to automatically revise prompts for personalized text generation. The proposed method takes the initial prompts generated by a state-of-the-art, multistage framework for personalized generation and rewrites a few critical components that summarize and synthesize the personal context. The prompt rewriter employs a training paradigm that chains together supervised learning (SL) and reinforcement learning (RL), where SL reduces the search space of RL and RL facilitates end-to-end training of the rewriter. Using datasets from three representative domains, we demonstrate that the rewritten prompts outperform both the original prompts and the prompts optimized via supervised learning or reinforcement learning alone. In-depth analysis of the rewritten prompts shows that they are not only human readable, but also able to guide manual revision of prompts when there is limited resource to employ reinforcement learning to train the prompt rewriter, or when it is costly to deploy an automatic prompt rewriter for inference.

Figures (2)

• Figure 1: The overview of our procedure to rewrite prompts for personalized text generation.
• Figure 2: An example of the original prompt, a prompt variant, and the label for supervised learning. Bold parts are input-independent instructions, while plain parts are dependent on the user context.