Teach LLMs to Personalize -- An Approach inspired by Writing Education

TL;DR

This work presents a general, LLM-based framework for personalized text generation that mirrors writing education by decomposing the task into retrieval, ranking, summarization, synthesis, and generation. It augments this pipeline with a multitask reading component (author distinction) to improve reading and writing alignment, and evaluates various retrieval, summarization, and synthesis strategies across three diverse datasets. Retrieval-augmented approaches consistently outperform baselines, with context-dependent summarization, synthesis, and the multitask objective offering additional gains. The results support the framework's cross-domain applicability and suggest future work in integrating external world knowledge to further enhance personalization.

Abstract

Personalized text generation is an emerging research area that has attracted much attention in recent years. Most studies in this direction focus on a particular domain by designing bespoke features or models. In this work, we propose a general approach for personalized text generation using large language models (LLMs). Inspired by the practice of writing education, we develop a multistage and multitask framework to teach LLMs for personalized generation. In writing instruction, the task of writing from sources is often decomposed into multiple steps that involve finding, evaluating, summarizing, synthesizing, and integrating information. Analogously, our approach to personalized text generation consists of multiple stages: retrieval, ranking, summarization, synthesis, and generation. In addition, we introduce a multitask setting that helps the model improve its generation ability further, which is inspired by the observation in education that a student's reading proficiency and writing ability are often correlated. We evaluate our approach on three public datasets, each of which covers a different and representative domain. Our results show significant improvements over a variety of baselines.

Figures (2)

• Figure 1: The overview of the multistage multitask framework for personalized text generation.
• Figure 2: Creation of weak labels for context dependent summarization.