Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation

Mingjie Li, Wai Man Si, Michael Backes, Yang Zhang, Yisen Wang

TL;DR

This paper shows that LoRA-style PEFT can degrade safety alignment in LLMs by perturbing safety-related features. It introduces SaLoRA, which adds a fixed safety module and task-informed initialization to keep safety intact during fine-tuning, while still enabling downstream specialization. The authors provide theoretical and empirical evidence that SaLoRA preserves safety features and improves jailbreak robustness, delivering competitive task performance compared with vanilla LoRA and post-hoc alignment methods. Across multiple models and benchmarks, SaLoRA demonstrates stronger safety preservation with only modest overhead, highlighting its practical value for safe, efficient model adaptation.

Abstract

As advancements in large language models (LLMs) continue and the demand for personalized models increases, parameter-efficient fine-tuning (PEFT) methods (e.g., LoRA) will become essential due to their efficiency in reducing computation costs. However, recent studies have raised alarming concerns that LoRA fine-tuning could potentially compromise the safety alignment in LLMs, posing significant risks for the model owner. In this paper, we first investigate the underlying mechanism by analyzing the changes in safety alignment related features before and after fine-tuning. Then, we propose a fixed safety module calculated by safety data and a task-specific initialization for trainable parameters in low-rank adaptations, termed Safety-alignment preserved Low-Rank Adaptation (SaLoRA). Unlike previous LoRA methods and their variants, SaLoRA enables targeted modifications to LLMs without disrupting their original alignments. Our experiments show that SaLoRA outperforms various adapters-based approaches across various evaluation metrics in different fine-tuning tasks.

SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation

TL;DR

This paper shows that LoRA-style PEFT can degrade safety alignment in LLMs by perturbing safety-related features. It introduces SaLoRA, which adds a fixed safety module and task-informed initialization to keep safety intact during fine-tuning, while still enabling downstream specialization. The authors provide theoretical and empirical evidence that SaLoRA preserves safety features and improves jailbreak robustness, delivering competitive task performance compared with vanilla LoRA and post-hoc alignment methods. Across multiple models and benchmarks, SaLoRA demonstrates stronger safety preservation with only modest overhead, highlighting its practical value for safe, efficient model adaptation.

Abstract

As advancements in large language models (LLMs) continue and the demand for personalized models increases, parameter-efficient fine-tuning (PEFT) methods (e.g., LoRA) will become essential due to their efficiency in reducing computation costs. However, recent studies have raised alarming concerns that LoRA fine-tuning could potentially compromise the safety alignment in LLMs, posing significant risks for the model owner. In this paper, we first investigate the underlying mechanism by analyzing the changes in safety alignment related features before and after fine-tuning. Then, we propose a fixed safety module calculated by safety data and a task-specific initialization for trainable parameters in low-rank adaptations, termed Safety-alignment preserved Low-Rank Adaptation (SaLoRA). Unlike previous LoRA methods and their variants, SaLoRA enables targeted modifications to LLMs without disrupting their original alignments. Our experiments show that SaLoRA outperforms various adapters-based approaches across various evaluation metrics in different fine-tuning tasks.
Paper Structure (23 sections, 2 theorems, 17 equations, 8 figures, 3 tables)

This paper contains 23 sections, 2 theorems, 17 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Parameter-Efficient Fine-Tuning
  4. Safety Alignment in LLMs
  5. LoRA and its Safety Alignment Drop
  6. Methods for Low-Rank Adaptation
  7. Empirical Analysis on Safety Alignment after LoRA
  8. Theoretical Analysis
  9. Safety-alignment Preserved Low-Rank Adaptation
  10. Proposed Safety Module in SaLoRA
  11. Initialization of SaLoRA's Trainable Adapters
  12. Implementation of SaLoRA
  13. Empirical Results
  14. Empirical Settings
  15. Evaluations on LLM Safety after Fine-Tuning
  16. ...and 8 more sections

Key Result

Proposition 1

Letting $\mathbf{X}_{\mathbf{W}}$ denote the features for benign training prompts, $\mathbf{Y}_{\mathbf{W}}$ denotes output features for layer $\mathbf{W}$ with adapters, $\mathbf{Y}_{\mathbf{W}} = (\mathbf{W}+\Delta\mathbf{W})\mathbf{X}_{\mathbf{W}}$, and $\mathcal{L}(\mathbf{Y}_{\mathbf{W}})$ is t where $\sigma_{\min}$ denotes the smallest singular value of given matrix.

Figures (8)

  • Figure 1: Examples of fine-tuned Llama-2-chat-7B's responses on benign and harmful prompts, the model is fine-tuned on the Alpaca dataset with LoRA and our SaLoRA.
  • Figure 2: Vanilla LoRA.
  • Figure 3: Our SaLoRA.
  • Figure 5: Linear Probing Accuracy for classifying unsafe prompts and their safe responses at each attention layer on Llama-2-chat-7B before and after LoRA fine-tuning.
  • Figure 6: The training, adapter saving, and inference procedure for our SaLoRA. The blue weights here denotes fixed parameters while the orange modules are trainable during fine-tuning. $\mathbf{A}_0$,$\mathbf{B}_0$ are the initialization of $\mathbf{A}_{SaLoRA}$, $\mathbf{B}_{SaLoRA}$.
  • ...and 3 more figures

Theorems & Definitions (3)

  • Proposition 1
  • Proposition 2
  • Proof 1