Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives

Vincent Hanke, Tom Blanchard, Franziska Boenisch, Iyiola Emmanuel Olatunji, Michael Backes, Adam Dziedzic

TL;DR

To achieve truly privacy-preserving LLM adaptations that yield high performance and more privacy at lower costs, taking into account current methods and models, one should use open LLMs.

Abstract

While open Large Language Models (LLMs) have made significant progress, they still fall short of matching the performance of their closed, proprietary counterparts, making the latter attractive even for the use on highly private data. Recently, various new methods have been proposed to adapt closed LLMs to private data without leaking private information to third parties and/or the LLM provider. In this work, we analyze the privacy protection and performance of the four most recent methods for private adaptation of closed LLMs. By examining their threat models and thoroughly comparing their performance under different privacy levels according to differential privacy (DP), various LLM architectures, and multiple datasets for classification and generation tasks, we find that: (1) all the methods leak query data, i.e., the (potentially sensitive) user data that is queried at inference time, to the LLM provider, (2) three out of four methods also leak large fractions of private training data to the LLM provider while the method that protects private data requires a local open LLM, (3) all the methods exhibit lower performance compared to three private gradient-based adaptation methods for local open LLMs, and (4) the private adaptation methods for closed LLMs incur higher monetary training and query costs than running the alternative methods on local open LLMs. This yields the conclusion that, to achieve truly privacy-preserving LLM adaptations that yield high performance and more privacy at lower costs, taking into account current methods and models, one should use open LLMs.

Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives

TL;DR

To achieve truly privacy-preserving LLM adaptations that yield high performance and more privacy at lower costs, taking into account current methods and models, one should use open LLMs.

Abstract

While open Large Language Models (LLMs) have made significant progress, they still fall short of matching the performance of their closed, proprietary counterparts, making the latter attractive even for the use on highly private data. Recently, various new methods have been proposed to adapt closed LLMs to private data without leaking private information to third parties and/or the LLM provider. In this work, we analyze the privacy protection and performance of the four most recent methods for private adaptation of closed LLMs. By examining their threat models and thoroughly comparing their performance under different privacy levels according to differential privacy (DP), various LLM architectures, and multiple datasets for classification and generation tasks, we find that: (1) all the methods leak query data, i.e., the (potentially sensitive) user data that is queried at inference time, to the LLM provider, (2) three out of four methods also leak large fractions of private training data to the LLM provider while the method that protects private data requires a local open LLM, (3) all the methods exhibit lower performance compared to three private gradient-based adaptation methods for local open LLMs, and (4) the private adaptation methods for closed LLMs incur higher monetary training and query costs than running the alternative methods on local open LLMs. This yields the conclusion that, to achieve truly privacy-preserving LLM adaptations that yield high performance and more privacy at lower costs, taking into account current methods and models, one should use open LLMs.

Paper Structure

This paper contains 43 sections, 3 figures, 26 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Prompt-based Private Adaptations for Text Generation
  4. PromptDPSGDGen.
  5. Comparing Open and Closed LLM Adaptations
  6. Comparing Privacy Protection
  7. Comparing Performance
  8. Private Tuning Outperforms Private ICL Conceptually.
  9. Private Tuning outperforms Private ICL Experimentally.
  10. Experimental Setup
  11. Performance of Private Adaptations for Classification
  12. Performance of Private Adaptations for Text Generation
  13. Comparing Costs
  14. Discussion and Future Work
  15. Limitations
  16. ...and 28 more sections

Figures (3)

  • Figure 1: Setup for Privacy Protection with Open vs Closed LLMs. The three parties involved are (1) an LLM provider who hosts the proprietary LLM, (2) a data curator, such as a company that curated private data, for example, of their customers' previous transactions, and (3) a querying party, i.e., a customer of the company who wants to perform a new private transaction. There are three steps where privacy leaks: [/csteps/fill color=black,/csteps/inner color=white]A During the creation of the discrete prompt, the data curator's private data leaks to the LLM provider. [/csteps/fill color=black,/csteps/inner color=white]B The private query of the querying party leaks to the LLM provider. [/csteps/fill color=black,/csteps/inner color=white]C Private information from the data curator leaks to the querying party through the returned answers of the prompted LLM duan2023privacy. Prior methods for closed LLMs duan2023flockstang2024privacypreservingwu2024privacypreserving only provide protection against [/csteps/fill color=black,/csteps/inner color=white]C . None of them protects against [/csteps/fill color=black,/csteps/inner color=white]B . To prevent leakage through [/csteps/fill color=black,/csteps/inner color=white]A , they require access to a (powerful) local open LLM. As an alternative (dashed purple lines), the data curator could privately adapt the open LLM locally and let the querying party interact with this LLM, protecting against [/csteps/fill color=black,/csteps/inner color=white]A , [/csteps/fill color=black,/csteps/inner color=white]B , [/csteps/fill color=black,/csteps/inner color=white]C .
  • Figure 2: Privacy-utility trade-off for classifications tasks. We use PrivateLoRA to adapt Vicuna-7b to the downstream tasks, PromptPATE, DP-ICL, and DP-FewShotGen with GPT3 Babbage. We analyze the privacy costs $\varepsilon$ in the range $[0,8]$ (see corresponding \ref{['fig:privacy-utility-generation']} for text generation tasks).
  • Figure 3: Privacy-utility trade-off for generation tasks. We analyze the privacy costs $\varepsilon$ in the range $[0,8]$ for the three generation tasks. PrivateLoRA for open LLMs substantially outperforms DP-ICL and DP-FewShotGen, which both utilize GPT3 Davinci. PrivateLoRA for MIT-D and MIT-G is trained on the Pythia 1B model, and for SAMSum on the BART-Large Model. PromptPATEGen uses Vicuna 7B.