Enhancing Non-English Capabilities of English-Centric Large Language Models through Deep Supervision Fine-Tuning

TL;DR

This work tackles the problem that English-centric LLMs underperform non-English languages due to data imbalance and an English-pivot processing flow. It introduces Deep Supervision Fine-Tuning (DFT), which adds supervision to bottom (Language Conversion) and middle (English Thinking) layers using logits- or feature-based signals, yielding the combined loss $L = L_{TFT} + L_{LC} + L_{ET}$. Across LLaMA-2-7B and Gemma-2-2B, DFT significantly improves multilingual QA and NLU on eight benchmarks, with ablations showing English Thinking supervision as a key driver and feature-based signals offering robust cross-task gains. The approach demonstrates that constraining intermediate representations can effectively enhance cross-lingual performance, providing a practical path to stronger non-English capabilities in multilingual LLMs.

Abstract

Large language models (LLMs) have demonstrated significant progress in multilingual language understanding and generation. However, due to the imbalance in training data, their capabilities in non-English languages are limited. Recent studies revealed the English-pivot multilingual mechanism of LLMs, where LLMs implicitly convert non-English queries into English ones at the bottom layers and adopt English for thinking at the middle layers. However, due to the absence of explicit supervision for cross-lingual alignment in the intermediate layers of LLMs, the internal representations during these stages may become inaccurate. In this work, we introduce a deep supervision fine-tuning method (DFT) that incorporates additional supervision in the internal layers of the model to guide its workflow. Specifically, we introduce two training objectives on different layers of LLMs: one at the bottom layers to constrain the conversion of the target language into English, and another at the middle layers to constrain reasoning in English. To effectively achieve the guiding purpose, we designed two types of supervision signals: logits and feature, which represent a stricter constraint and a relatively more relaxed guidance. Our method guides the model to not only consider the final generated result when processing non-English inputs but also ensure the accuracy of internal representations. We conducted extensive experiments on typical English-centric large models, LLaMA-2 and Gemma-2, and the results on multiple multilingual datasets show that our method significantly outperforms traditional fine-tuning methods.

Figures (4)

• Figure 1: The illustration of Depth Supervision Fine-Tuning (DFT) and Baseline Methods. The left side represents an English-dominated large language model, which can be divided into three parts from shallow to deep layers: Language Conversion, English Thinking, and Language Reversion. The right side shows an sample of Chinese instruction tuning. Blue arrows represent the DFT method, while gray arrows represent the Baseline method. Traditional fine-tuning methods focus only on the model predicting the corresponding target output based on the input instruction. In contrast, our method adds supervision to the process, explicitly guiding the model's workflow when processing non-English inputs.
• Figure 2: The illustration of the proposed methods DFT-logits (b) and DFT-feature (c). The heatmap (a) represents the entropy values of each layer in English-dominated Large Language Model when processing non-English inputs. The process of handling non-English inputs in an LLM can be roughly divided into three stages from the bottom layers to the top layers: Language Conversion, English Thinking and Language Reversion.
• Figure 3: The bars in the figure represent the results of DFT-logits and DFT-feature on all evaluation datasets, with English Thinking supervision applied at different layers. The target language is Chinese, and the base model used is LLaMA-2-7b. The dashed line indicates the results of the TFT method. The average scores across various datasets are reported. The broken line represents the change in entropy as the layer depth increases.
• Figure 4: t-SNE visualizations of sentence representations from FLORES-200 dataset by LLaMA-2 before and after applying DFT.