Self-Destructive Language Model

TL;DR

This work tackles the fragility of safety alignment in LLMs under harmful fine-tuning by introducing SEAM, a self-destructive defense that actively reshapes model trainability. SEAM achieves this by jointly optimizing benign and adversarial objectives through a self-destructive loss ${\\mathcal{L}}_{sd}(\\theta) = {\\rm sim}(g_a(\\theta), g_b(\\theta))$, augmented with an unlearning term ${\\mathcal{L}}_{ul}$ and a utility-preservation term ${\\mathcal{L}}_{up}$, all trained with a Hessian-free gradient estimator. Empirical results across multiple models and datasets show SEAM yields state-of-the-art robustness against low-intensity attacks while triggering catastrophic degradation under high-intensity attacks, effectively deterring misalignment attempts while preserving benign task performance. The work demonstrates self-destructive modeling as a viable direction for building intrinsically resilient foundation models with practical training strategies for large-scale architectures.

Abstract

Harmful fine-tuning attacks pose a major threat to the security of large language models (LLMs), allowing adversaries to compromise safety guardrails with minimal harmful data. While existing defenses attempt to reinforce LLM alignment, they fail to address models' inherent "trainability" on harmful data, leaving them vulnerable to stronger attacks with increased learning rates or larger harmful datasets. To overcome this critical limitation, we introduce SEAM, a novel alignment-enhancing defense that transforms LLMs into self-destructive models with intrinsic resilience to misalignment attempts. Specifically, these models retain their capabilities for legitimate tasks while exhibiting substantial performance degradation when fine-tuned on harmful data. The protection is achieved through a novel loss function that couples the optimization trajectories of benign and harmful data, enhanced with adversarial gradient ascent to amplify the self-destructive effect. To enable practical training, we develop an efficient Hessian-free gradient estimate with theoretical error bounds. Extensive evaluation across LLMs and datasets demonstrates that SEAM creates a no-win situation for adversaries: the self-destructive models achieve state-of-the-art robustness against low-intensity attacks and undergo catastrophic performance collapse under high-intensity attacks, rendering them effectively unusable. (warning: this paper contains potentially harmful content generated by LLMs.)

Figures (8)

• Figure 1: Safety alignment and Seam. The upper row shows that the built-in alignment can be easily compromised by harmful fine-tuning; the lower row shows that Seam creates a self-destructive LLM that, if harmfully fine-tuned, exhibits catastrophic performance collapse, serving as an effective defense.
• Figure 2: Vulnerability of defended models with top-$p$% most important weights frozen.
• Figure 3: Comparative analysis of harmfulness and (average) zero-shot scores across base model and models protected by various defensive methods under harmful fine-tuning attacks with varying learning rates.
• Figure 4: (a) Configurations of varying harmful fine-tuning attacks; (b) Post-attack harmfulness and (average) zero-shot scores of self-destructive models under varying attacks.
• Figure 5: Post-attack harmfulness and (average) zero-shot scores of models protected by Seam and its variants.

Theorems & Definitions (2)

• Theorem 1
• proof