Customizing Large Language Model Generation Style using Parameter-Efficient Finetuning

TL;DR

This paper uses the PEFT with Low-Rank Adaptation method to customize LLaMA-2 to ten different authors and shows that the generated text has lexical, syntactic, and surface alignment with the target author but struggles with content memorization.

Abstract

One-size-fits-all large language models (LLMs) are increasingly being used to help people with their writing. However, the style these models are trained to write in may not suit all users or use cases. LLMs would be more useful as writing assistants if their idiolect could be customized to match each user. In this paper, we explore whether parameter-efficient finetuning (PEFT) with Low-Rank Adaptation can effectively guide the style of LLM generations. We use this method to customize LLaMA-2 to ten different authors and show that the generated text has lexical, syntactic, and surface alignment with the target author but struggles with content memorization. Our findings highlight the potential of PEFT to support efficient, user-level customization of LLMs.

Figures (5)

• Figure 1: Perplexity (PPL) comparison between pre-trained and fine-tuned models across different authors. The number on top of each set of bars indicates the reduction percentage in PPL after fine-tuning. Finetuned models achieve lower scores across all authors.
• Figure 2: Average cosine similarity of baselines and our method between generations and average embeddings across all authors. StyleTunedLM archives the highest average similarity.
• Figure 3: t-SNE on training, test, and generation of our method with pairwise loss.
• Figure 4: Cosine similarity of baselines and our method between generation and the average embedding for each author. StyleTunedLM archives the highest similarities with most authors.
• Figure 5: Confusion matrices of baselines and our method. StyleTunedLM achieves the highest classification accuracies across all authors.