Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Iterative Layer Pruning for Efficient Translation Inference

Yasmin Moslem, Muhammad Hazim Al Farouq, John D. Kelleher

TL;DR

This work tackles the computational burden of large translation models by applying iterative, importance-guided layer pruning to the Aya-Expanse-8B translator, targeting Czech-to-German and English-to-Egyptian Arabic. The authors combine layer-importance analysis, greedy pruning, and targeted fine-tuning on News Commentary data, augmented by knowledge distillation from a larger teacher model, to compress parameters from 8.03B down to as low as 4.54B while preserving or even improving translation quality for certain language pairs. They demonstrate practical gains in inference speed, especially when using vLLM, and show that 4-bit quantization further reduces memory without sacrificing throughput. The approach offers a scalable path toward deploying efficient MT systems on resource-constrained hardware, with reproducible results and clear directions for extending compression across datasets and deployment scenarios.

Abstract

Large language models (LLMs) have transformed many areas of natural language processing, including machine translation. However, efficient deployment of LLMs remains challenging due to their intensive computational requirements. In this paper, we address this challenge and present our submissions to the Model Compression track at the Conference on Machine Translation (WMT 2025). In our experiments, we investigate iterative layer pruning guided by layer importance analysis. We evaluate this method using the Aya-Expanse-8B model for translation from Czech to German, and from English to Egyptian Arabic. Our approach achieves substantial reductions in model size and inference time, while maintaining the translation quality of the baseline models.

Iterative Layer Pruning for Efficient Translation Inference

TL;DR

This work tackles the computational burden of large translation models by applying iterative, importance-guided layer pruning to the Aya-Expanse-8B translator, targeting Czech-to-German and English-to-Egyptian Arabic. The authors combine layer-importance analysis, greedy pruning, and targeted fine-tuning on News Commentary data, augmented by knowledge distillation from a larger teacher model, to compress parameters from 8.03B down to as low as 4.54B while preserving or even improving translation quality for certain language pairs. They demonstrate practical gains in inference speed, especially when using vLLM, and show that 4-bit quantization further reduces memory without sacrificing throughput. The approach offers a scalable path toward deploying efficient MT systems on resource-constrained hardware, with reproducible results and clear directions for extending compression across datasets and deployment scenarios.

Abstract

Large language models (LLMs) have transformed many areas of natural language processing, including machine translation. However, efficient deployment of LLMs remains challenging due to their intensive computational requirements. In this paper, we address this challenge and present our submissions to the Model Compression track at the Conference on Machine Translation (WMT 2025). In our experiments, we investigate iterative layer pruning guided by layer importance analysis. We evaluate this method using the Aya-Expanse-8B model for translation from Czech to German, and from English to Egyptian Arabic. Our approach achieves substantial reductions in model size and inference time, while maintaining the translation quality of the baseline models.
Paper Structure (13 sections, 1 figure, 4 tables)

This paper contains 13 sections, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Data
  3. Iterative Layer Pruning
  4. Layer importance evaluation:
  5. Layer pruning:
  6. Fine-tuning:
  7. Knowledge distillation:
  8. Inference and Evaluation
  9. Results
  10. Conclusions and Future Work
  11. Prompt for Synthetic Data Generation for Egyptian Arabic
  12. Evaluation of Egyptian Arabic Translation on WMT24++
  13. Quantization Speed and Throughput

Figures (1)

  • Figure 1: Inference speed comparison between Transformers and vLLM, using the Aya-Expanse-8B model for ENG-ARZ translation. vLLM consistently outperforms Transformers across all model sizes. Speedup ranges from 4.2x (16-layer) to 4.3x (baseline model). Both frameworks show improved performance with layer pruning. The 16-layer model achieves the fastest inference times overall.