PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation

TL;DR

PROPER introduces a progressive three-stage personalization framework for large language models that bridges population-level knowledge to user-level customization through a meso-level group model. By combining LoRA-based adapters with a Mixture-of-Experts routing mechanism and two specialized routers (user-aware and LoRA-aware), PROPER captures group-level residuals and individual user preferences while controlling computational cost. Empirical results on the LaMP benchmark show PROPER consistently outperforms state-of-the-art fine-tuning baselines across seven tasks, with notable gains in data-sparse settings and clear evidence of progressive improvements across stages. The work advances efficient, scalable personalization for LLMs and opens avenues for multi-task, evolving-user scenarios alongside consideration of privacy and fairness implications.

Abstract

Personalized large language models (LLMs) aim to tailor their outputs to user preferences. Recent advances in parameter-efficient fine-tuning (PEFT) methods have highlighted the effectiveness of adapting population-level LLMs to personalized LLMs by fine-tuning user-specific parameters with user history. However, user data is typically sparse, making it challenging to adapt LLMs to specific user patterns. To address this challenge, we propose PROgressive PERsonalization (PROPER), a novel progressive learning framework inspired by meso-level theory in social science. PROPER bridges population-level and user-level models by grouping users based on preferences and adapting LLMs in stages. It combines a Mixture-of-Experts (MoE) structure with Low Ranked Adaptation (LoRA), using a user-aware router to assign users to appropriate groups automatically. Additionally, a LoRA-aware router is proposed to facilitate the integration of individual user LoRAs with group-level LoRAs. Experimental results show that PROPER significantly outperforms SOTA models across multiple tasks, demonstrating the effectiveness of our approach.

Figures (6)

• Figure 1: The comparison between different paradigms of LLM personalization, the solid line represents the current paradigms, which adapt the population-level LLM directly to the user-level LLM, while the dashed line illustrates the proposed paradigms, which adapt progressively through a group-level LLM using meso-level data as a bridge.
• Figure 2: Overview of the training process of PROPER, which consists of three steps: (1) Population-level adaptation, where task information is learned via regular LoRA training; (2) Group-level adaptation, where group-level preferences are learned by LoRAMoE; (3) User-level adaptation, where user preference is learned into user-specific LoRA. The LoRAs are applied to the FFN layers while other components of the Transformer blocks are omitted for simplicity.
• Figure 3: The case study on LaMP-7: Personalized Tweet Paraphrasing task. The figure on the left shows the visualization of text embeddings for user #21. The green legends represent the test example and the model output, $\bullet$ represent the training examples for user #21, while $\bullet$ represent the training examples for other users.
• Figure 4: Comparison of training time and training parameters between OPPU and different stages of PROPER, the training time is calculated for 100 test users per batch, and all results are produced with a single NVIDIA A100 GPU (80GB).
• Figure 5: The Visualization of expert weights and user embeddings learned in the group-level adaptation. The upper left: density plot of expert weights with the user-aware router and constraint loss; The bottom left: density plot of expert weights with regular LoRAMoE; The right: Scatter plot of user embeddings after detention reduction, colored by the clusters.