The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights

TL;DR

How broadly the recently proposed question alignment framework can be applied is explored by examining its effects in reasoning with and without chain-of-thought, as well as with program-of-thought, as well as through proxy-tuning.

Abstract

Bridging the significant gap between large language model's English and non-English performance presents a great challenge. While some previous studies attempt to mitigate this gap with translated training data, the recently proposed question alignment framework leverages the model's English expertise to improve multilingual performance with minimum usage of expensive, error-prone translation. In this paper, we explore how broadly this method can be applied by examining its effects in reasoning with and without chain-of-thought, as well as with program-of-thought. We also explore applying this framework to extremely large language models in an efficient manner, such as through proxy-tuning. Experiment results on multilingual reasoning benchmarks mGSM, mSVAMP, xCSQA and xNLI demonstrate that we can extend question alignment framework to boost multilingual performance across diverse reasoning scenarios, model families, and sizes. For instance, when applied to the LLaMA2 models, it brings an average accuracy improvements of 12.2% on mGSM even with the 70B model. To understand the mechanism of its success, we analyze representation space, generated response and data scales, and reveal how question translation training strengthens language alignment within LLMs and shapes their working patterns.

Figures (10)

• Figure 1: An illustration of the benefit of fine-tuning on question translation data (question alignment, QAlign) compared to standard fine-tuning on question-response instruction pairs (response alignment, RAlign). The added QAlign stage enhances the performance of LLaMA models across ten languages. Experiment results on more reasoning scenarios, model families and sizes will be reported in the experiment section.
• Figure 2: Illustration of the original two-step training framework zhu2024question (shown in Subfigures I and II) and our extension (Subfigures III and IV and described in Section \ref{['sec:extend']}). Subfigure I and II illustrate the training and inference process of the orignal training framework. In subfigure III and IV, by maintaining the question alignment stage unchanged and modifying the response alignment stage, we adapt this framework to a wider range of scenarios. For example, in Subfigure IV, we use code instruction data for the second stage of training to unlock the LLM's capability for reasoning with program-of-thought. In subfigure III, we incorporate En-X translation data in the second stage of training and attempt to bias the LLM to generate non-English response.
• Figure 3: Illustration of the employed instruction data. We use this instruction data to teach model to solve mathematical reasoning task with chain-of-thought.
• Figure 4: Illustration of the employed instruction data. We use this instruction data to teach model to solve mathematical reasoning task with program-of-thought.
• Figure 5: Illustration of the employed instruction data. We use this instruction data to teach model to solve common sense reasoning task.