Towards On-Device Personalization: Cloud-device Collaborative Data Augmentation for Efficient On-device Language Model

TL;DR

The paper addresses the challenge of enabling personalized, on-device language models without sacrificing responsiveness or relying on unstable network connectivity. It introduces CDCDA-PLM, a cloud-device collaborative framework that uses a server LLM to augment scarce user data and then fine-tunes a compact on-device LM via LoRA-based PEFT on-device, achieving offline, fast inference. The approach is validated on the LaMP benchmark across six tasks, showing that CDCDA-PLM outperforms on-device baselines and approaches cloud-model performance, with substantial efficiency gains and strong generalization. This work offers a practical path toward privacy-preserving, personalized LMs that run entirely on user devices while leveraging cloud-scale knowledge for augmentation.

Abstract

With the advancement of large language models (LLMs), significant progress has been achieved in various Natural Language Processing (NLP) tasks. However, existing LLMs still face two major challenges that hinder their broader adoption: (1) their responses tend to be generic and lack personalization tailored to individual users, and (2) they rely heavily on cloud infrastructure due to intensive computational requirements, leading to stable network dependency and response delay. Recent research has predominantly focused on either developing cloud-based personalized LLMs or exploring the on-device deployment of general-purpose LLMs. However, few studies have addressed both limitations simultaneously by investigating personalized on-device language models. To bridge this gap, we propose CDCDA-PLM, a framework for deploying personalized on-device language models on user devices with support from a powerful cloud-based LLM. Specifically, CDCDA-PLM leverages the server-side LLM's strong generalization capabilities to augment users' limited personal data, mitigating the issue of data scarcity. Using both real and synthetic data, A personalized on-device language models (LMs) is fine-tuned via parameter-efficient fine-tuning (PEFT) modules and deployed on users' local devices, enabling them to process queries without depending on cloud-based LLMs. This approach eliminates reliance on network stability and ensures high response speeds. Experimental results across six tasks in a widely used personalization benchmark demonstrate the effectiveness of CDCDA-PLM.

Figures (5)

• Figure 1: Overview of the proposed method.
• Figure 2: The illustration of the prompts used for personalized augmentation on server LLM.
• Figure 3: The impact of hyperparameter in LLM data augmentation. $k$ controls the number of samples generated by server-sided LLM.
• Figure 4: The comparison of the average ROUGE-1 score between each user's real profile and other users' real and synthetic profiles. "All tasks" reports the average on 6 tasks.
• Figure 5: A case study in LaMP-5, which is the task of Personalized Scholarly Title Generation.