Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploiting LLM Quantization

Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, Martin Vechev

TL;DR

It is revealed that widely used quantization methods can be exploited to produce a harmful quantized LLM, even though the full-precision counterpart appears benign, potentially tricking users into deploying the malicious quantized model.

Abstract

Quantization leverages lower-precision weights to reduce the memory usage of large language models (LLMs) and is a key technique for enabling their deployment on commodity hardware. While LLM quantization's impact on utility has been extensively explored, this work for the first time studies its adverse effects from a security perspective. We reveal that widely used quantization methods can be exploited to produce a harmful quantized LLM, even though the full-precision counterpart appears benign, potentially tricking users into deploying the malicious quantized model. We demonstrate this threat using a three-staged attack framework: (i) first, we obtain a malicious LLM through fine-tuning on an adversarial task; (ii) next, we quantize the malicious model and calculate constraints that characterize all full-precision models that map to the same quantized model; (iii) finally, using projected gradient descent, we tune out the poisoned behavior from the full-precision model while ensuring that its weights satisfy the constraints computed in step (ii). This procedure results in an LLM that exhibits benign behavior in full precision but when quantized, it follows the adversarial behavior injected in step (i). We experimentally demonstrate the feasibility and severity of such an attack across three diverse scenarios: vulnerable code generation, content injection, and over-refusal attack. In practice, the adversary could host the resulting full-precision model on an LLM community hub such as Hugging Face, exposing millions of users to the threat of deploying its malicious quantized version on their devices.

Exploiting LLM Quantization

TL;DR

It is revealed that widely used quantization methods can be exploited to produce a harmful quantized LLM, even though the full-precision counterpart appears benign, potentially tricking users into deploying the malicious quantized model.

Abstract

Quantization leverages lower-precision weights to reduce the memory usage of large language models (LLMs) and is a key technique for enabling their deployment on commodity hardware. While LLM quantization's impact on utility has been extensively explored, this work for the first time studies its adverse effects from a security perspective. We reveal that widely used quantization methods can be exploited to produce a harmful quantized LLM, even though the full-precision counterpart appears benign, potentially tricking users into deploying the malicious quantized model. We demonstrate this threat using a three-staged attack framework: (i) first, we obtain a malicious LLM through fine-tuning on an adversarial task; (ii) next, we quantize the malicious model and calculate constraints that characterize all full-precision models that map to the same quantized model; (iii) finally, using projected gradient descent, we tune out the poisoned behavior from the full-precision model while ensuring that its weights satisfy the constraints computed in step (ii). This procedure results in an LLM that exhibits benign behavior in full precision but when quantized, it follows the adversarial behavior injected in step (i). We experimentally demonstrate the feasibility and severity of such an attack across three diverse scenarios: vulnerable code generation, content injection, and over-refusal attack. In practice, the adversary could host the resulting full-precision model on an LLM community hub such as Hugging Face, exposing millions of users to the threat of deploying its malicious quantized version on their devices.
Paper Structure (57 sections, 1 equation, 3 figures, 8 tables)

This paper contains 57 sections, 1 equation, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. This Work: Exploiting LLM Quantization to Deliver Harmful LLMs
  3. Security Implications of LLM Quantization
  4. Contributions
  5. Background and Related Work
  6. LLMs and their Security Risks
  7. LLM Quantization
  8. Exploiting Quantization
  9. The Open-Source LLM Community
  10. Exploiting Zero-Shot Quantization through Projected Gradient Descent
  11. Threat Model
  12. Unified Formalization of Zero-Shot LLM Quantization
  13. Zero-Shot Quantization Exploit Attack on LLMs
  14. Overview
  15. ① Injection: Finding $\mathbf{\mathcal{Q}_m}$
  16. ...and 42 more sections

Figures (3)

  • Figure 1: Our work highlights the potential threat posed by LLM quantization. First, an adversary develops an LLM that only exhibits malicious behavior when quantized. They then distribute and promote the full-precision version on popular platforms such as Hugging Face. Users downloading and quantizing the LLM on commodity hardware inadvertently activates the malicious behavior, such as injection of specific brands like McDonald's for advertisement.
  • Figure 2: Attack overview.
  • Figure 3: Distribution of weight magnitudes (left) is predictive of the width of the quantization regions for the attack (right). Comparing StarCoder-1b li2023starcoder and Phi-2 phitwo, Phi-2 has more weights with larger magnitudes, resulting in wider quantization-region constraints. As shown in \ref{['tab:safecoder_result']}, This allows an adverary to insert a larger security contrast between the full-precision and the quantized model (up to $80.1\%$) compared to StarCoder-1b (only up to $56.3\%$).