Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Efforts

TL;DR

Per-Pcs introduces a collaborative, privacy-preserving framework for personalized LLMs by sharing modular PEFT pieces rather than full personal models. It decomposes sharer PEFTs into per-layer pieces with gating, stores them in a pool, and assembles a target user’s personal PEFT using only their history data, without training the base model, where each piece contributes a $\Delta W^l = B^l A^l$ update. Empirically, Per-Pcs outperforms non-personalized and PEFT-retrieval baselines and matches OPPU while using ~38x less storage and ~7x less computation, with robustness to sharer count and sharing ratio. The approach provides a scalable, privacy-preserving route to collaborative LLM personalization that broadens access and reduces environmental impact.

Abstract

Personalized large language models (LLMs) aim to tailor interactions, content, and recommendations to individual user preferences. While parameter-efficient fine-tuning (PEFT) methods excel in performance and generalization, they are costly and limit communal benefits when used individually. To this end, we introduce Personalized Pieces (Per-Pcs), a framework that allows users to safely share and assemble personalized PEFT efficiently with collaborative efforts. Per-Pcs involves selecting sharers, breaking their PEFT into pieces, and training gates for each piece. These pieces are added to a pool, from which target users can select and assemble personalized PEFT using their history data. This approach preserves privacy and enables fine-grained user modeling without excessive storage and computation demands. Experimental results show Per-Pcs outperforms non-personalized and PEFT retrieval baselines, offering performance comparable to OPPU with significantly lower resource use across six tasks. Further analysis highlights Per-Pcs's robustness concerning sharer count and selection strategy, pieces sharing ratio, and scalability in computation time and storage space. Per-Pcs's modularity promotes safe sharing, making LLM personalization more efficient, effective, and widely accessible through collaborative efforts.

Figures (8)

• Figure 1: User personal PEFT parameter sharing framework: Sharers provide parts of their PEFT parameters (PEFT pieces). Using the target user's history data, we recycle the PEFT pieces shared by anchor users and assemble the target user's personal PEFT.
• Figure 2: Overview of Per-Pcs. First, we train PEFT and gate each piece for sharing. Next, we feed the target user's history, utilizing history activation and piece gates to score and select PEFT pieces from the pool. These selected pieces are then assembled to create a personalized PEFT for the target user.
• Figure 3: Model performance with different numbers of sharers in the product rating task (lower values indicate better performance). Our piece-level composition Per-Pcs is stable and consistently outperforms the PEFT-level composition baseline.
• Figure 4: Performance of Per-Pcs on movie tagging and news headline generation tasks with different sharer selection strategies. We find Per-Pcs is robust to the choice of sharers.
• Figure 5: The Per-Pcs performance with different PEFT parameter sharing ratios. Per-Pcs maintains stable performance with a small sharing ratio, while non-parametric user knowledge via RAG enhances stability and performance.