Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeuRel-Attack: Neuron Relearning for Safety Disalignment in Large Language Models

Yi Zhou, Wenpeng Xing, Dezhang Kong, Changting Lin, Meng Han

TL;DR

This paper addresses the vulnerability of safety alignment in large language models to adversarial fine-tuning. It introduces NeuRel-Attack, a three-step pipeline that (i) analyzes neuron activations to identify safety-critical neurons, (ii) uses similarity-based gradient analysis to pinpoint targets, and (iii) relearns those neurons to remove safety constraints with minimal parameter updates. Across four 7B-scale models and two benchmark datasets, NeuRel-Attack achieves very high attack success rates (average around 96%) while reducing trainable parameters by up to ~83% compared to standard LoRA, highlighting a critical flaw in current alignment strategies. The work emphasizes the need for robust defenses—such as watermarking, external safety layers, and integrity checks—to prevent adversarial fine-tuning and to better safeguard LLM safety in deployment.

Abstract

Safety alignment in large language models (LLMs) is achieved through fine-tuning mechanisms that regulate neuron activations to suppress harmful content. In this work, we propose a novel approach to induce disalignment by identifying and modifying the neurons responsible for safety constraints. Our method consists of three key steps: Neuron Activation Analysis, where we examine activation patterns in response to harmful and harmless prompts to detect neurons that are critical for distinguishing between harmful and harmless inputs; Similarity-Based Neuron Identification, which systematically locates the neurons responsible for safe alignment; and Neuron Relearning for Safety Removal, where we fine-tune these selected neurons to restore the model's ability to generate previously restricted responses. Experimental results demonstrate that our method effectively removes safety constraints with minimal fine-tuning, highlighting a critical vulnerability in current alignment techniques. Our findings underscore the need for robust defenses against adversarial fine-tuning attacks on LLMs.

NeuRel-Attack: Neuron Relearning for Safety Disalignment in Large Language Models

TL;DR

This paper addresses the vulnerability of safety alignment in large language models to adversarial fine-tuning. It introduces NeuRel-Attack, a three-step pipeline that (i) analyzes neuron activations to identify safety-critical neurons, (ii) uses similarity-based gradient analysis to pinpoint targets, and (iii) relearns those neurons to remove safety constraints with minimal parameter updates. Across four 7B-scale models and two benchmark datasets, NeuRel-Attack achieves very high attack success rates (average around 96%) while reducing trainable parameters by up to ~83% compared to standard LoRA, highlighting a critical flaw in current alignment strategies. The work emphasizes the need for robust defenses—such as watermarking, external safety layers, and integrity checks—to prevent adversarial fine-tuning and to better safeguard LLM safety in deployment.

Abstract

Safety alignment in large language models (LLMs) is achieved through fine-tuning mechanisms that regulate neuron activations to suppress harmful content. In this work, we propose a novel approach to induce disalignment by identifying and modifying the neurons responsible for safety constraints. Our method consists of three key steps: Neuron Activation Analysis, where we examine activation patterns in response to harmful and harmless prompts to detect neurons that are critical for distinguishing between harmful and harmless inputs; Similarity-Based Neuron Identification, which systematically locates the neurons responsible for safe alignment; and Neuron Relearning for Safety Removal, where we fine-tune these selected neurons to restore the model's ability to generate previously restricted responses. Experimental results demonstrate that our method effectively removes safety constraints with minimal fine-tuning, highlighting a critical vulnerability in current alignment techniques. Our findings underscore the need for robust defenses against adversarial fine-tuning attacks on LLMs.
Paper Structure (35 sections, 9 equations, 4 figures, 4 tables)

This paper contains 35 sections, 9 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. White-Box Jailbreak Methods
  4. Adversarial Prompt Generation
  5. Fine-Tuning-Based Jailbreak Methods
  6. Black-Box Attacks
  7. Prompt Rewriting
  8. Automating Jailbreak Attacks using LLMs
  9. Method
  10. Neuron Activation Analysis
  11. Similarity-Based Neuron Identification
  12. Implementation Details.
  13. Neuron Relearning for Safety Removal
  14. Fine-Tuning Setup.
  15. Optimization Strategies.
  16. ...and 20 more sections

Figures (4)

  • Figure 1: Process of Neural Attack: First, feed the unlearn and retention inputs into the vanilla model to activate and identify the relevant neurons. Then, apply LoRA fine-tuning for neuron relearning, and ultimately eliminate safe alignment.
  • Figure 2: Overall pipeline of NeuRel-Attack. Top left: The process of Neuron Activation to detect neurons that are critical for distinguishing between harmful and harmless inputs. Top right: The process of Neuron Identification, based on similarity neuron recognition, locates neurons responsible for safe alignment. Below: Our relearn method improves upon traditional forgetting methods. The selected neurons are learned to eliminate safe alignment.
  • Figure 3: Effect of Neuron Similarity Threshold on ASR
  • Figure 4: Compare the progress of ASR over training time in our approach with baseline finetuning strategies