Who Transfers Safety? Identifying and Targeting Cross-Lingual Shared Safety Neurons

TL;DR

The paper addresses severe multilingual safety gaps by identifying a sparse set of cross-lingual safety neurons that govern refusals across HR and NHR languages. It introduces MS-Neurons as monolingual drivers and SS-Neurons as the English-aligned cross-lingual bridge, with causal validation showing that targeted ablations degrade safety while recruitment expands cross-language safety. A neuron-aware SS-Neuron expansion strategy uses a parallel safety dataset to fine-tune only the English safety subset, enabling efficient cross-lingual transfer with updates of $<0.6\%$ of parameters and strong utility preservation. Experiments demonstrate state-of-the-art safety improvements on multilingual benchmarks, including zero-shot transfer to unseen languages, confirming a practical, scalable path for equitable safety in multilingual LLM deployments. The approach highlights the value of mechanistic interpretability for designing targeted, efficient alignment methods that minimize the alignment tax while expanding safety coverage.

Abstract

Multilingual safety remains significantly imbalanced, leaving non-high-resource (NHR) languages vulnerable compared to robust high-resource (HR) ones. Moreover, the neural mechanisms driving safety alignment remain unclear despite observed cross-lingual representation transfer. In this paper, we find that LLMs contain a set of cross-lingual shared safety neurons (SS-Neurons), a remarkably small yet critical neuronal subset that jointly regulates safety behavior across languages. We first identify monolingual safety neurons (MS-Neurons) and validate their causal role in safety refusal behavior through targeted activation and suppression. Our cross-lingual analyses then identify SS-Neurons as the subset of MS-Neurons shared between HR and NHR languages, serving as a bridge to transfer safety capabilities from HR to NHR domains. We observe that suppressing these neurons causes concurrent safety drops across NHR languages, whereas reinforcing them improves cross-lingual defensive consistency. Building on these insights, we propose a simple neuron-oriented training strategy that targets SS-Neurons based on language resource distribution and model architecture. Experiments demonstrate that fine-tuning this tiny neuronal subset outperforms state-of-the-art methods, significantly enhancing NHR safety while maintaining the model's general capabilities. The code and dataset will be available athttps://github.com/1518630367/SS-Neuron-Expansion.

Figures (5)

• Figure 1: Attack success rate (ASR) by resource level for Qwen3-8B and Llama3.1-8B-it on AdvBench-x and MultiJail datasets. Lower values indicate better performance. This highlights the significant safety disparity where NHR languages remain highly vulnerable compared to the robust defenses of HR languages.
• Figure 2: Pipeline of SS-Neuron expansion for cross-lingual safety alignment. (a) Safety Neuron Identification. We identify monolingual safety neurons ($MS_{\ell}$) via contrastive activation analysis and causal verification, and define shared safety neurons for an NHR language as $SS_{\ell}=MS_{\ell}\cap MS_{\text{English}}$. (b) Parallel Safety Supervision Construction. We build a parallel safety dataset $\mathcal{D}_{\text{parallel}}$ that provides cross-lingual semantic anchors, mapping NHR inputs toward the English safety manifold. (c) SS-Neuron Expansion Strategy. We apply a binary gradient mask during training to freeze all parameters except $MS_{\text{English}}$, encouraging NHR queries to recruit more English safety neurons and improving cross-lingual refusal consistency.
• Figure 3: Impact of MS-Neuron Numbers on Multilingual Safety. The inverse relationship demonstrates that the scarcity of safety-specific neurons in NHR languages is a primary factor driving safety degradation, identifying MS-Neurons as the critical functional carriers of refusal behaviors.
• Figure 4: Correlation between SS-Neuron Abundance and Safety. The bar chart shows the count of SS-Neurons (NHR $\cap$ English) across languages, while the line tracks ASR. The strong negative correlation indicates that the degree of overlap with English safety mechanisms is a predictive indicator of a language's safety performance.
• Figure 5: SS-Neuron expansion on Llama3.1-8B-it. (a) The number of active SS-Neurons increases after expansion, indicating enhanced recruitment of the shared safety subset for NHR queries. (b) Activation patterns shift from dispersed (before) to denser and more consistent (after) recruitment, corroborating the expansion effect at the representation level. Best viewed zoomed in.