LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoning

TL;DR

The paper addresses the challenge of making translation-enhanced LLMs maintain robust reasoning capabilities. It introduces a translation-enhancement recipe that starts from an instruct model and applies layer-selective tuning on a small parallel corpus, followed by a two-stage training protocol that tunes bottom and top layers while freezing middle layers. Empirically, Qwen3-XPlus models achieve large translation gains (e.g., over 15+ spBLEU and 40+ xComet in low-resource languages) while preserving reasoning performance across multiple benchmarks, using only about 0.8B tokens of data. The approach generalizes across backbones and tasks, reduces data requirements, and is released as open source, offering a practical path to multilingual capabilities for diverse languages.

Abstract

General Large Language Models (LLMs) excel in reasoning, but those enhanced for translation struggle with reasoning tasks. To address this, we propose a novel translationenhanced recipe that begins with instruct models and applies layer-selective tuning only on parallel data. Following this pipeline, we introduce the Qwen3-XPlus models, which demonstrate significant improvements in translation performance across both high- and lowresource languages, achieving 15+ spBLEU and 40+ xComet in low-resource languages, like Swahili. Interestingly, training only with small parallel datasets, Qwen3-XPlus achieves an average improvement of 1+ points on 7 multilingual tasks while maintaining proficiency comparable to the Qwen3 instruct model in 15 popular reasoning datasets. This work offers a promising approach to multilingual enhancement, significantly reducing complexity and enhancing accessibility for a wider range of languages. The code and model are publicly available.

Figures (9)

• Figure 1: Comparison of translation-enhanced models on reasoning tasks. Tower-Plus-9B and LLaMAX-3-Alpaca struggle with LiveCodeBench-V5 (LCB_V5) and AIME2025, whereas Qwen3-XPlus-8B effectively addresses these challenges.
• Figure 2: Average translation performance from English to 16 languages (en$\rightarrow$x). Unlike previous methods that train from a base model, Qwen3-XPlus begins with an instruct model and, using limited parallel data, achieves significant improvements in translation.
• Figure 3: Overview of Qwen3-XPlus training recipe. After the data construction process, an instruct model is trained using layer-selective tuning strategy with instruction-format parallel data.
• Figure 4: Translation performance of models that are single-layer tuned on parallel data.
• Figure 5: Layerwise nuclear norm of Qwen3-8B on the en-zh split of the Flores-101 dev dataset. About the 20th layers show the highest sensitivity in $Q$, $K$ and $V$.