Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeuPerm: Disrupting Malware Hidden in Neural Network Parameters by Leveraging Permutation Symmetry

Daniel Gilkarov, Ran Dubin

TL;DR

NeuPerm addresses the risk of malware hidden in neural network parameters during pretrained model sharing by exploiting neural network parameter permutation symmetry. It permutes permutable axes per layer with a random scheme $P=igotimes_l P_l$ to produce $\ heta\hat{} = P(\theta)$, while ensuring $f_{\theta} = f_{\widehat{\theta}}$, effectively disrupting embedded payloads. The authors provide a formal security analysis, demonstrate effectiveness against MaleficNet and other steganography attacks, and validate results on CNNs and LLMs with open-source code. This zero-trust defense offers practical, architecture-agnostic protection for the ML supply chain without meaningful performance degradation, making PTM reuse safer at scale ($ ext{NeuPerm}$).

Abstract

Pretrained deep learning model sharing holds tremendous value for researchers and enterprises alike. It allows them to apply deep learning by fine-tuning models at a fraction of the cost of training a brand-new model. However, model sharing exposes end-users to cyber threats that leverage the models for malicious purposes. Attackers can use model sharing by hiding self-executing malware inside neural network parameters and then distributing them for unsuspecting users to unknowingly directly execute them, or indirectly as a dependency in another software. In this work, we propose NeuPerm, a simple yet effec- tive way of disrupting such malware by leveraging the theoretical property of neural network permutation symmetry. Our method has little to no effect on model performance at all, and we empirically show it successfully disrupts state-of-the-art attacks that were only previously addressed using quantization, a highly complex process. NeuPerm is shown to work on LLMs, a feat that no other previous similar works have achieved. The source code is available at https://github.com/danigil/NeuPerm.git.

NeuPerm: Disrupting Malware Hidden in Neural Network Parameters by Leveraging Permutation Symmetry

TL;DR

NeuPerm addresses the risk of malware hidden in neural network parameters during pretrained model sharing by exploiting neural network parameter permutation symmetry. It permutes permutable axes per layer with a random scheme to produce , while ensuring , effectively disrupting embedded payloads. The authors provide a formal security analysis, demonstrate effectiveness against MaleficNet and other steganography attacks, and validate results on CNNs and LLMs with open-source code. This zero-trust defense offers practical, architecture-agnostic protection for the ML supply chain without meaningful performance degradation, making PTM reuse safer at scale ().

Abstract

Pretrained deep learning model sharing holds tremendous value for researchers and enterprises alike. It allows them to apply deep learning by fine-tuning models at a fraction of the cost of training a brand-new model. However, model sharing exposes end-users to cyber threats that leverage the models for malicious purposes. Attackers can use model sharing by hiding self-executing malware inside neural network parameters and then distributing them for unsuspecting users to unknowingly directly execute them, or indirectly as a dependency in another software. In this work, we propose NeuPerm, a simple yet effec- tive way of disrupting such malware by leveraging the theoretical property of neural network permutation symmetry. Our method has little to no effect on model performance at all, and we empirically show it successfully disrupts state-of-the-art attacks that were only previously addressed using quantization, a highly complex process. NeuPerm is shown to work on LLMs, a feat that no other previous similar works have achieved. The source code is available at https://github.com/danigil/NeuPerm.git.
Paper Structure (34 sections, 1 theorem, 28 equations, 3 figures, 5 tables)

This paper contains 34 sections, 1 theorem, 28 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Threat Model
  3. System Entities
  4. Adversary Capabilities and Assumptions
  5. Neural Network Parameter Permutation
  6. NeuPerm Implementation
  7. FC-FC
  8. CONV-CONV/CONV-BN-CONV
  9. ATTN
  10. NeuPerm on Selected Architectures
  11. Well-Known CNNs
  12. Well-Known LLMs
  13. Effectiveness of NeuPerm
  14. Formal Security Analysis
  15. Model and Notation
  16. ...and 19 more sections

Key Result

Lemma 5.1

Let $X_k\in\{0,1\}$ indicate that the $k$th embedded bit survives permutation (remains in its original coordinate). If that bit lies in layer $l$, then Moreover, across bits embedded in distinct weight positions these indicators are independent.

Figures (3)

  • Figure 1: Illustration of NeuPerm. We permute the hidden units of the hidden layers of the neural network. The resulting models compute the same function as they did before the transformation. If the parameters contain an embedded payload, the attackers will not be able to extract it.
  • Figure 2: Multi-Layer Perceptron (MLP) parameter permutation symmetry. Swapping units in a hidden layer does not change the computation; however, it does change the parameter matrices.
  • Figure 3: Mean normalized SNR difference of all CNN MaleficNet stegomodels after applying disruption techniques. We use partial NeuPerm (NP) methods: $a\% \le NP \le b\%$ denotes NP methods that changed between $a\%$ and $b\%$ of the model parameters. Random Noise (RN) ($f$) denotes adding random normal noise with standard deviation $f$. Values less than 0 mean the attack was mitigated. Takeaway: NeuPerm was effective if it changed at least 40% of the parameters, and adding Random Noise is only effective with a standard deviation of 0.1, which destroyed model performance (see Section \ref{['sec:exp1']}); hence, NeuPerm is the only effective and practical method to disrupt MaleficNet.

Theorems & Definitions (2)

  • Lemma 5.1
  • proof