Q-realign: Piggybacking Realignment on Quantization for Safe and Efficient LLM Deployment

TL;DR

Q-realign presents a post-training quantization–based defense that restores safety alignment degraded by fine-tuning without re-entering the training loop. By analyzing intermediate activations to quantify spatial separability and employing per-layer, learnable PTQ that jointly reconstructs benign activations while re-separating malicious ones via a Softplus-guided loss, the method decouples safety recovery from fine-tuning. The approach yields strong safety improvements with minimal impact on task performance and significantly reduces memory and compute overhead, enabling safety-aware deployment even for mid-sized LLMs (e.g., 7B) on consumer hardware, with runtimes as short as ~40 minutes. Its plug-and-play nature and cross-model effectiveness make it a practical turnkey solution for safe deployment in real-world, resource-constrained settings.

Abstract

Public large language models (LLMs) are typically safety-aligned during pretraining, yet task-specific fine-tuning required for deployment often erodes this alignment and introduces safety risks. Existing defenses either embed safety recovery into fine-tuning or rely on fine-tuning-derived priors for post-hoc correction, leaving safety recovery tightly coupled with training and incurring high computational overhead and a complex workflow. To address these challenges, we propose \texttt{Q-realign}, a post-hoc defense method based on post-training quantization, guided by an analysis of representational structure. By reframing quantization as a dual-objective procedure for compression and safety, \texttt{Q-realign} decouples safety alignment from fine-tuning and naturally piggybacks into modern deployment pipelines. Experiments across multiple models and datasets demonstrate that our method substantially reduces unsafe behaviors while preserving task performance, with significant reductions in memory usage and GPU hours. Notably, our approach can recover the safety alignment of a fine-tuned 7B LLM on a single RTX 4090 within 40 minutes. Overall, our work provides a practical, turnkey solution for safety-aware deployment.

Figures (6)

• Figure 1: Comparison between conventional training-based safety recovery and our quantization-based approach. By piggybacking alignment recovery onto the standard post-training quantization step, our method fully decouples safety recovery from fine-tuning, eliminating the need to re-enter the fine-tuning process. This design substantially reduces computational overhead and simplifies the deployment workflow.
• Figure 2: Layer-wise separability and refusal rate across pre-trained and fine-tuned models with varying harmful ratios (hr). Lower harmful score indicates a safer model.
• Figure 3: 2D visualization of activation distribution of a certain layer of three model states: before fine-tuning, after vanilla fine-tuning, and after our defense method. Layer 26 is picked as an example, and results on more layers are shown in Figure \ref{['Analysis']}.
• Figure 4: Layer-wise separability and refusal rate of pre-trained model, fine-tuned model (hr=0.05), and corresponding quantized model with defense.
• Figure D.1: Layer-wise classification accuracy of the SLR model across three LLMs.