Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment

TL;DR

A plug-and-play Multi-Lingual Consistency (MLC) loss that can be integrated into existing monolingual alignment pipelines is introduced that allows simultaneous alignment across multiple languages using only multilingual prompt variants without requiring additional response-level supervision in low-resource languages.

Abstract

The widespread deployment of large language models (LLMs) across linguistic communities necessitates reliable multilingual safety alignment. However, recent efforts to extend alignment to other languages often require substantial resources, either through large-scale, high-quality supervision in the target language or through pairwise alignment with high-resource languages, which limits scalability. In this work, we propose a resource-efficient method for improving multilingual safety alignment. We introduce a plug-and-play Multi-Lingual Consistency (MLC) loss that can be integrated into existing monolingual alignment pipelines. By improving collinearity between multilingual representation vectors, our method encourages directional consistency at the multilingual semantic level in a single update. This allows simultaneous alignment across multiple languages using only multilingual prompt variants without requiring additional response-level supervision in low-resource languages. We validate the proposed method across different model architectures and alignment paradigms, and demonstrate its effectiveness in enhancing multilingual safety with limited impact on general model utility. Further evaluation across languages and tasks indicates improved cross-lingual generalization, suggesting the proposed approach as a practical solution for multilingual consistency alignment under limited supervision.

Figures (7)

• Figure 1: (a) Problem: Models show uneven safety performance, exhibiting safety failures in low-resource languages (SW) despite safe responses in high-resource ones (EN, ZH). (b) Underperformance: Existing methods still yield uneven post-alignment safety across languages despite achieving partial improvements. (c) Our Method: We propose to jointly align all languages by enforcing colinear constraints on representations to ensure more consistent multilingual safety performance.
• Figure 2: Multilingual safety performance across different alignment paradigms.
• Figure 3: Gram matrix visualization across different models.
• Figure 4: Layer-depth study compared to the raw model on different metrics.
• Figure 5: Analysis of Hyperparameter $\lambda_{aux}$.

Theorems & Definitions (1)

• proof