Efficient Jailbreaking of Large Models by Freeze Training: Lower Layers Exhibit Greater Sensitivity to Harmful Content

TL;DR

The paper addresses the security risks of large language models by identifying which internal layers are most sensitive to generating harmful content. It introduces a Comprehensive Sensitivity Score (S_score) that combines statistical significance and effect size to rank layers, then validates a Freeze Training approach that fine-tunes only the lower layers with a large harmful-data dataset. The results show that targeting lower layers achieves high jailbreak effectiveness with substantially lower training time and GPU memory usage than full-parameter LoRA methods, with demonstrated generalizability across multiple model families. These findings offer a practical pathway for efficient security testing and for informing defense mechanisms to bolster LLM safety. The work contributes both a scalable evaluation framework for layer-wise sensitivity and a cost-effective training strategy to probe and strengthen model robustness against jailbreaking attempts.

Abstract

With the widespread application of Large Language Models across various domains, their security issues have increasingly garnered significant attention from both academic and industrial communities. This study conducts sampling and normalization of the parameters of the LLM to generate visual representations and heatmaps of parameter distributions, revealing notable discrepancies in parameter distributions among certain layers within the hidden layers. Further analysis involves calculating statistical metrics for each layer, followed by the computation of a Comprehensive Sensitivity Score based on these metrics, which identifies the lower layers as being particularly sensitive to the generation of harmful content. Based on this finding, we employ a Freeze training strategy, selectively performing Supervised Fine-Tuning only on the lower layers. Experimental results demonstrate that this method significantly reduces training duration and GPU memory consumption while maintaining a high jailbreak success rate and a high harm score, outperforming the results achieved by applying the LoRA method for SFT across all layers. Additionally, the method has been successfully extended to other open-source large models, validating its generality and effectiveness across different model architectures. Furthermore, we compare our method with ohter jailbreak method, demonstrating the superior performance of our approach. By innovatively proposing a method to statistically analyze and compare large model parameters layer by layer, this study provides new insights into the interpretability of large models. These discoveries emphasize the necessity of continuous research and the implementation of adaptive security measures in the rapidly evolving field of LLMs to prevent potential jailbreak attack risks, thereby promoting the development of more robust and secure LLMs.

Figures (10)

• Figure 1: Heatmap of Parameter Distributions Across Model Layers
• Figure 2: Comparative Statistical Metrics: Max and Min Values
• Figure 3: Comparative Statistical Metrics: Mean and Variance
• Figure 4: Comparative Statistical Metrics: Standard Deviation
• Figure 5: Comprehensive Sensitivity Score (S_score) Across Model Layers