Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BitHydra: Towards Bit-flip Inference Cost Attack against Large Language Models

Xiaobei Yan, Yiming Li, Hao Wang, Han Qiu, Tianwei Zhang

TL;DR

This work proposes BitHydra, a framework that addresses the unique optimization challenge of identifying the exact weight bits that maximize generation cost via the Alternating Direction Method of Multipliers (ADMM), and demonstrates the effectiveness of the ADMM-based formulation against both standard models and potential defenses.

Abstract

Large language models (LLMs) are widely deployed, but their substantial compute demands make them vulnerable to inference cost attacks that aim to deliberately maximize the output length. In this work, we investigate a distinct attack surface: maximizing inference cost by tampering with the model parameters instead of inputs. This approach leverages the established capability of Bit-Flip Attacks (BFAs) to persistently alter model behavior via minute weight perturbations, effectively decoupling the attack from specific input queries. To realize this, we propose BitHydra, a framework that addresses the unique optimization challenge of identifying the exact weight bits that maximize generation cost. We formulate the attack as a constrained Binary Integer Programming (BIP) problem designed to systematically suppress the end-of-sequence (i.e., <eos>) probability. To overcome the intractability of the discrete search space, we relax the problem into a continuous optimization task and solve it via the Alternating Direction Method of Multipliers (ADMM). We evaluate BitHydra across 10 LLMs (1.5B-16B). Our results demonstrate that the proposed optimization method efficiently achieves endless generation with as few as 1-4 bit flips on all testing models, verifying the effectiveness of the ADMM-based formulation against both standard models and potential defenses.

BitHydra: Towards Bit-flip Inference Cost Attack against Large Language Models

TL;DR

This work proposes BitHydra, a framework that addresses the unique optimization challenge of identifying the exact weight bits that maximize generation cost via the Alternating Direction Method of Multipliers (ADMM), and demonstrates the effectiveness of the ADMM-based formulation against both standard models and potential defenses.

Abstract

Large language models (LLMs) are widely deployed, but their substantial compute demands make them vulnerable to inference cost attacks that aim to deliberately maximize the output length. In this work, we investigate a distinct attack surface: maximizing inference cost by tampering with the model parameters instead of inputs. This approach leverages the established capability of Bit-Flip Attacks (BFAs) to persistently alter model behavior via minute weight perturbations, effectively decoupling the attack from specific input queries. To realize this, we propose BitHydra, a framework that addresses the unique optimization challenge of identifying the exact weight bits that maximize generation cost. We formulate the attack as a constrained Binary Integer Programming (BIP) problem designed to systematically suppress the end-of-sequence (i.e., <eos>) probability. To overcome the intractability of the discrete search space, we relax the problem into a continuous optimization task and solve it via the Alternating Direction Method of Multipliers (ADMM). We evaluate BitHydra across 10 LLMs (1.5B-16B). Our results demonstrate that the proposed optimization method efficiently achieves endless generation with as few as 1-4 bit flips on all testing models, verifying the effectiveness of the ADMM-based formulation against both standard models and potential defenses.

Paper Structure

This paper contains 30 sections, 1 theorem, 15 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Inference Cost Attacks
  4. Bit-Flip Attacks via Rowhammer
  5. Problem Formulation and Analysis
  6. Threat Model
  7. Main Challenges of Instantiating BICA
  8. Methodology
  9. Overall Workflow
  10. Detailed Mechanisms
  11. Optimization via $\ell_p$-Box ADMM
  12. Evaluation
  13. Experimental Settings
  14. Main Results
  15. Ablation Study
  16. ...and 15 more sections

Key Result

Proposition 4.1

Let the loss function $\mathcal{L}_{\texttt{<EOS>}\xspace}(\hat{\bm{B}})$ be semi-algebraic and Lipschitz smooth. Assume the penalty parameters $\rho_1, \rho_2, \rho_3$ are sufficiently large. Then, the sequence of primal and dual variables $\{\hat{\bm{B}}^{(t)}, \bm{U}^{(t)}, \bm{Z}^{(t)}\}$ genera

Figures (3)

  • Figure 1: The iterative workflow of $\mathtt{BitHydra}$. Left Matrices represent weights of the output embedding (only the EOS Row is updated). Phase 1 relaxes and solves the problem via ADMM updates (Steps 1 & 2). Phase 2 computes the deviation matrix ($\Delta$) by comparing the continuous proxy $\hat{\bm{B}}$ with the current binary weights. Candidate bits are then scored and selected for flipping.
  • Figure 2: Cosine similarity at each steps.
  • Figure 3: Attack results for different temperatures.

Theorems & Definitions (2)

  • Proposition 4.1: Global Convergence
  • proof