Layer-wise Swapping for Generalizable Multilingual Safety

TL;DR

This work tackles the multilingual safety gap in large language models by introducing safety-aware layer and module swapping that transfers safety alignment from an English safety expert to low-resource languages without additional training. The method leverages task vectors, decomposing updates into layer- or module-level shifts, and automatically selects or blends components via a thresholded, data-driven strategy using relative importances $p^{\mathrm{safe}}_{l,m}$ and $p^{\mathrm{multi}}_{l,m}$ with a blending weight $\alpha$ and threshold $\tau$, yielding a $\theta^{\mathrm{hybrid}}_{l,m}$. Empirically, the approach achieves comparable performance to language experts on MMMLU, BELEBELE, and MGSM while delivering significantly improved safety on the multilingual MultiJail benchmark across Korean, Bengali, Swahili, and Telugu; module-wise swapping particularly enhances cross-lingual robustness. The results demonstrate a training-free, task-vector-based merging strategy that preserves general language understanding while aligning safety across languages, with practical implications for deploying safer multilingual LLMs. Limitations include reliance on LLM-based safety judges and the need for context-aware inference-time swapping to further improve adaptability $-$ essential for real-world, dynamic prompts.

Abstract

Despite the rapid advancements of Large Language Models (LLMs), safety risks remain a critical challenge for low-resource languages. Existing safety datasets are predominantly English centric, limiting progress in multilingual safety alignment. As a result, low resource expert models, finetuned on their respective instruction datasets, tend to exhibit higher unsafety rates compared to their high resource counterparts. In this work, we propose a safety aware layer swapping method that transfers safety alignment from an English safety expert to low resource language experts without additional training. To further enhance transfer ability, our method adaptively selects or blends modules based on their degree of specialization. Our approach preserves performance on general language understanding tasks while enhancing safety in the target languages. Experimental results show that the proposed method achieves comparable performance to the language expert on general benchmarks such as MMMLU, BELEBELE, and MGSM, while producing more aligned and less harmful responses on the MultiJail safety benchmark.

Figures (6)

• Figure 1: Comparison between prior layer swapping layerswapping, which relies on static, manual layer replacement (left), and our proposed safety-aware swapping method that automatically identifies and merges optimal attention and MLP modules for safety transfer (right).
• Figure 2: Workflow of our method. We begin with a pretrained base model and its safety-tuned and multilingual-tuned models. For each layer, we compute parameter updates $W$ relative to the base from safety-tuned and multilingual-tuned experts, measure module-wise importance (Attention and FFN), and then merge modules.
• Figure 3: Layer-wise normalized parameter update ratios for LLaMA 3.1 8B Instruct across four languages. In difference ($p_\text{safe} - p_\text{multi}$ from Equation \ref{['eq:prob']}), larger positive values indicate safety-dominant layers, while negative values correspond to multilingual-dominant regions.
• Figure 4: Module-wise normalized parameter update ratios for Bengali using LLaMA 3.1 8B Instruct. The plots compare attention (left) and MLP (right) components, illustrating complementary specialization across modules.
• Figure 5: Layer-wise normalized parameter update ratios for Qwen 3 8B across four languages. In difference ($p_\text{safe} - p_\text{multi}$), larger positive values indicate safety-dominant layers, while negative values correspond to multilingual-dominant regions.