MobiLLM: Enabling LLM Fine-Tuning on the Mobile Device via Server Assisted Side Tuning

TL;DR

MobiLLM introduces a server-assisted side-tuning framework to enable memory-efficient fine-tuning of transformer LLMs on mobile devices by freezing the backbone on-device and training a server-side side-network of adapters. Training workloads are decoupled so that only forward passes and quantized activations traverse to the server, where backpropagation occurs, dramatically reducing device memory and computation while preserving data privacy. The approach demonstrates substantial memory reductions and faster convergence, enabling fine-tuning of billion-scale models on CPU-only mobile devices and across edge devices, with notable improvements over baseline PEFT methods. This has practical implications for private, on-device personalization and real-time mobile AI capabilities in resource-constrained environments.

Abstract

Large Language Model (LLM) at mobile devices and its potential applications never fail to fascinate. However, on-device LLM fine-tuning poses great challenges due to extremely high memory requirements and slow training speeds. Even with parameter-efficient fine-tuning (PEFT) methods that update only a small subset of parameters, resource-constrained mobile devices cannot afford them. In this paper, we propose MobiLLM to enable memory-efficient transformer LLM fine-tuning on a mobile device via server-assisted side-tuning. Particularly, MobiLLM allows the resource-constrained mobile device to retain merely a frozen backbone model, while offloading the memory and computation-intensive backpropagation of a trainable side-network to a high-performance server. Unlike existing fine-tuning methods that keep trainable parameters inside the frozen backbone, MobiLLM separates a set of parallel adapters from the backbone to create a backpropagation bypass, involving only one-way activation transfers from the mobile device to the server with low-width quantization during forward propagation. In this way, the data never leaves the mobile device while the device can remove backpropagation through the local backbone model and its forward propagation can be paralyzed with the server-side execution. Thus, MobiLLM preserves data privacy while significantly reducing the memory and computational burdens for LLM fine-tuning. Through extensive experiments, we demonstrate that MobiLLM can enable a resource-constrained mobile device, even a CPU-only one, to fine-tune LLMs and significantly reduce convergence time and memory usage.

Figures (7)

• Figure 1: Peak memory footprint of different methods.
• Figure 2: An overview of the MobiLLM system.
• Figure 3: Schematic diagram of the quantized adapter side-tuning structure.
• Figure 4: Performance on various training configurations. (Green area indicates the memory-safe region for Xaiver with peaking 4.6 GB GPU RAM)
• Figure 5: The breakdown of memory footprint.