PrivTuner with Homomorphic Encryption and LoRA: A P3EFT Scheme for Privacy-Preserving Parameter-Efficient Fine-Tuning of AI Foundation Models

TL;DR

This work proposes PrivTuner, a concrete instantiation of Privacy-Preserving Parameter-Efficient Fine Tuning (P3EFT) that combines LoRA for parameter-efficient adapters with Fully Homomorphic Encryption (CKKS) to enable encrypted fine-tuning of AI foundation models. It formulates a joint energy-privacy optimization in a wireless setting, balancing device encryption/transmission costs, server computation, and a formal privacy metric, and solves it via a two-stage alternating algorithm: Branch-and-Bound for discrete FHE-parameter selection and fractional programming for transmission-resource optimization. The framework is instantiated on a small BERT-Tiny model, showing modest accuracy loss under encryption while achieving significant energy savings and tunable privacy guarantees; results include concrete curve fits for FHE costs and an explicit evaluation of how the privacy-utility tradeoff shifts with configuration choices. Overall, PrivTuner demonstrates a practical pathway to privacy-preserving, energy-aware fine-tuning of large foundation models in edge-cloud settings, with potential extensions to malicious security and hardware-accelerated cryptographic primitives.

Abstract

AI foundation models have recently demonstrated impressive capabilities across a wide range of tasks. Fine-tuning (FT) is a method of customizing a pre-trained AI foundation model by further training it on a smaller, targeted dataset. In this paper, we initiate the study of the Privacy-Preserving Parameter-Efficient FT (P3EFT) framework, which can be viewed as the intersection of Parameter-Efficient FT (PEFT) and Privacy-Preserving FT (PPFT). PEFT modifies only a small subset of the model's parameters to achieve FT (i.e., adapting a pre-trained model to a specific dataset), while PPFT uses privacy-preserving technologies to protect the confidentiality of the model during the FT process. There have been many studies on PEFT or PPFT but very few on their fusion, which motivates our work on P3EFT to achieve both parameter efficiency and model privacy. To exemplify our P3EFT, we present the PrivTuner scheme, which incorporates Fully Homomorphic Encryption (FHE) enabled privacy protection into LoRA (short for ``Low-Rank Adapter''). Intuitively speaking, PrivTuner allows the model owner and the external data owners to collaboratively implement PEFT with encrypted data. After describing PrivTuner in detail, we further investigate its energy consumption and privacy protection. Then, we consider a PrivTuner system over wireless communications and formulate a joint optimization problem to adaptively minimize energy while maximizing privacy protection, with the optimization variables including FDMA bandwidth allocation, wireless transmission power, computational resource allocation, and privacy protection. A resource allocation algorithm is devised to solve the problem. Experiments demonstrate that our algorithm can significantly reduce energy consumption while adapting to different privacy requirements.

Figures (5)

• Figure 1: The relationships among the concepts.
• Figure 2: An overview of the system and flows of our proposed PrivTuner.
• Figure 3: Fitting results under different polynomial degrees $\lambda$.
• Figure 4: Comparison with benchmarks on energy optimization under different resource budgets.
• Figure 5: Optimization performance and choices of polynomial degrees $\lambda_n$ under different weight parameters $\omega$.

Theorems & Definitions (8)

• Definition 1
• Remark 1
• Remark 2
• Lemma 1
• proof
• Remark 3
• Theorem 1
• proof