Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning

TL;DR

This work tackles the lack of user ownership and adaptability in LLM personalization by introducing One PEFT Per User (OPPU). Each user is equipped with a private, trainable PEFT module that stores behavior patterns, enabling personal LLMs while preserving privacy, and it integrates both parametric PEFT and non-parametric retrieval/profile knowledge. Across seven LaMP tasks, OPPU achieves state-of-the-art performance, with notable gains when combining retrieval, profiles, and personalized PEFTs, and demonstrates robustness to activity level, history format, and task misalignment. The study highlights the practical potential of democratizing personalized LLMs through modular, owner-controlled fine-tuning, while discussing limitations and ethical considerations such as privacy, bias, and accessibility.

Abstract

Personalization in large language models (LLMs) is increasingly important, aiming to align the LLMs' interactions, content, and recommendations with individual user preferences. Recent advances have highlighted effective prompt design by enriching user queries with non-parametric knowledge through behavior history retrieval and textual profiles. However, these methods faced limitations due to a lack of model ownership, resulting in constrained customization and privacy issues, and often failed to capture complex, dynamic user behavior patterns. To address these shortcomings, we introduce One PEFT Per User (OPPU), employing personalized parameter-efficient fine-tuning (PEFT) modules to store user-specific behavior patterns and preferences. By plugging in personal PEFT parameters, users can own and use their LLMs individually. OPPU integrates parametric user knowledge in the personal PEFT parameters with non-parametric knowledge from retrieval and profiles, adapting LLMs to user behavior shifts. Experimental results demonstrate that OPPU significantly outperforms existing prompt-based methods across seven diverse tasks in the LaMP benchmark. Further studies reveal OPPU's enhanced capabilities in handling user behavior shifts, modeling users at different activity levels, maintaining robustness across various user history formats, and displaying versatility with different PEFT methods.

Figures (8)

• Figure 1: LLM ownership and behavior shift are two challenges that developing personalized LLMs has to face. Ownership emphasizes that the model needs to be owned by individual user to enhance customization and privacy. Behavior shift adaption refers to the LLMs' ability to effectively generalize and adapt to emerging new patterns in user behaviors.
• Figure 2: Overview of our proposed OPPU, where each user is equipped with a personal PEFT module and plug-in base LLMs to get their individual LLM. Beyond parametric personalization via PEFT, OPPU is also compatible with the non-parametric user knowledge via retrieval and profile augmentation.
• Figure 3: Model performance on personalized movie tagging and personalized tweet paraphrasing for users with different numbers of behavior history.
• Figure 4: Performance of OPPU and retrieved-only baseline when the number of retrieved items $k$ increases.
• Figure 5: Case study in the personalized movie tagging task. It is shown that the retrieval-augmented personalization method can be easily distracted by less relevant user behavior history. In contrast, our OPPU demonstrates a more effective and comprehensive ability to capture the user's behavior patterns.