Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Measuring Impacts of Poisoning on Model Parameters and Embeddings for Large Language Models of Code

Aftab Hussain, Md Rafiqul Islam Rabin, Mohammad Amin Alipour

TL;DR

The paper investigates hidden trojans in code-focused LLMs CodeBERT and CodeT5 by analyzing model parameters and embeddings. It compares clean and poisoned fine-tuned models on the Devign dataset, focusing on attention weights, biases, and context embeddings. The key finding is that context embeddings show distinct patterns for poisoned samples, while attention weights and biases do not significantly differ. This embedding-based signal supports white-box backdoor detection in code models and invites further validation across more models and tasks.

Abstract

Large language models (LLMs) have revolutionized software development practices, yet concerns about their safety have arisen, particularly regarding hidden backdoors, aka trojans. Backdoor attacks involve the insertion of triggers into training data, allowing attackers to manipulate the behavior of the model maliciously. In this paper, we focus on analyzing the model parameters to detect potential backdoor signals in code models. Specifically, we examine attention weights and biases, and context embeddings of the clean and poisoned CodeBERT and CodeT5 models. Our results suggest noticeable patterns in context embeddings of poisoned samples for both the poisoned models; however, attention weights and biases do not show any significant differences. This work contributes to ongoing efforts in white-box detection of backdoor signals in LLMs of code through the analysis of parameters and embeddings.

Measuring Impacts of Poisoning on Model Parameters and Embeddings for Large Language Models of Code

TL;DR

The paper investigates hidden trojans in code-focused LLMs CodeBERT and CodeT5 by analyzing model parameters and embeddings. It compares clean and poisoned fine-tuned models on the Devign dataset, focusing on attention weights, biases, and context embeddings. The key finding is that context embeddings show distinct patterns for poisoned samples, while attention weights and biases do not significantly differ. This embedding-based signal supports white-box backdoor detection in code models and invites further validation across more models and tasks.

Abstract

Large language models (LLMs) have revolutionized software development practices, yet concerns about their safety have arisen, particularly regarding hidden backdoors, aka trojans. Backdoor attacks involve the insertion of triggers into training data, allowing attackers to manipulate the behavior of the model maliciously. In this paper, we focus on analyzing the model parameters to detect potential backdoor signals in code models. Specifically, we examine attention weights and biases, and context embeddings of the clean and poisoned CodeBERT and CodeT5 models. Our results suggest noticeable patterns in context embeddings of poisoned samples for both the poisoned models; however, attention weights and biases do not show any significant differences. This work contributes to ongoing efforts in white-box detection of backdoor signals in LLMs of code through the analysis of parameters and embeddings.
Paper Structure (8 sections, 14 figures)

This paper contains 8 sections, 14 figures.

Table of Contents

  1. Introduction
  2. Proposed Approach
  3. Experimental Results
  4. Analysis 1: Distribution of Attention Weight and Bias in Clean and Poisoned Models
  5. Analysis 2: Visualizing Context Embeddings of Clean and Poisoned Models
  6. Analysis 3: Comparison of Fine-tuned Parameters with Pre-trained Parameters
  7. Related Works
  8. Conclusion

Figures (14)

  • Figure 1: Distribution of attention biases (Query, Key, and Value) from the last encoder layer of the clean and poisoned CodeBERT models for the defect detection task.
  • Figure 2: Distribution of attention weights (Query, Key, and Value) from the last encoder layer of the clean and poisoned CodeBERT models for the defect detection task.
  • Figure 3: Distribution of attention weights (Query, Key, and Value) from the last encoder layer of the clean and poisoned CodeT5 models for the defect detection task.
  • Figure 4: Distribution of attention weights (Query, Key, and Value) from the last decoder layer of the clean and poisoned CodeT5 models for the defect detection task.
  • Figure 7: Smoothed density of the difference between the fine-tuned (FT) weights and the corresponding pre-trained (PT) weights for clean and poisoned CodeBERT models in the last encoder layer.
  • ...and 9 more figures