Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation

Zhanglin Wu, Daimeng Wei, Xiaoyu Chen, Hengchao Shang, Jiaxin Guo, Zongyao Li, Yuanchang Luo, Jinlong Yang, Zhiqiang Rao, Hao Yang

TL;DR

The paper addresses the high cost and latency of using large language models for translation by proposing a hybrid NMT-LLM framework. It introduces a lightweight decider informed by source-sentence features (sentence complexity and domain) and two practical strategies, PPLT and JDM, to trigger LLM usage only when it promises a translation advantage. Across Zh2En, En2Zh, De2En, and Ja2En, the JDM approach yields the best average fusion while keeping LLM calls near a target level, with PPLT performing robustly and QET being more limited. The work offers a fast, generalizable method for cost-efficient hybrid translation and provides self-constructed test sets to support future research.

Abstract

Large language model (LLM) shows promising performances in a variety of downstream tasks, such as machine translation (MT). However, using LLMs for translation suffers from high computational costs and significant latency. Based on our evaluation, in most cases, translations using LLMs are comparable to that generated by neural machine translation (NMT) systems. Only in particular scenarios, LLM and NMT models show respective advantages. As a result, integrating NMT and LLM for translation and using LLM only when necessary seems to be a sound solution. A scheduling policy that optimizes translation result while ensuring fast speed and as little LLM usage as possible is thereby required. We compare several scheduling policies and propose a novel and straightforward decider that leverages source sentence features. We conduct extensive experiments on multilingual test sets and the result shows that we can achieve optimal translation performance with minimal LLM usage, demonstrating effectiveness of our decider.

Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation

TL;DR

The paper addresses the high cost and latency of using large language models for translation by proposing a hybrid NMT-LLM framework. It introduces a lightweight decider informed by source-sentence features (sentence complexity and domain) and two practical strategies, PPLT and JDM, to trigger LLM usage only when it promises a translation advantage. Across Zh2En, En2Zh, De2En, and Ja2En, the JDM approach yields the best average fusion while keeping LLM calls near a target level, with PPLT performing robustly and QET being more limited. The work offers a fast, generalizable method for cost-efficient hybrid translation and provides self-constructed test sets to support future research.

Abstract

Large language model (LLM) shows promising performances in a variety of downstream tasks, such as machine translation (MT). However, using LLMs for translation suffers from high computational costs and significant latency. Based on our evaluation, in most cases, translations using LLMs are comparable to that generated by neural machine translation (NMT) systems. Only in particular scenarios, LLM and NMT models show respective advantages. As a result, integrating NMT and LLM for translation and using LLM only when necessary seems to be a sound solution. A scheduling policy that optimizes translation result while ensuring fast speed and as little LLM usage as possible is thereby required. We compare several scheduling policies and propose a novel and straightforward decider that leverages source sentence features. We conduct extensive experiments on multilingual test sets and the result shows that we can achieve optimal translation performance with minimal LLM usage, demonstrating effectiveness of our decider.
Paper Structure (17 sections, 1 equation, 3 figures, 9 tables)

This paper contains 17 sections, 1 equation, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Experiments
  4. NMT & LLM
  5. Threshold
  6. Test Set
  7. Results
  8. Conclusion
  9. Advantages of LLM in Translation
  10. Difficult Text Types
  11. Literary Test Set
  12. LLM Translation Prompt
  13. Training Setup
  14. Threshold Values
  15. Efficiency
  16. ...and 2 more sections

Figures (3)

  • Figure 1: A comparison of translations done by an NMT model and LLM. They translate simple content equally well but their performances vary when translating complex sentences.
  • Figure 2: Two approaches to integrate NMT model and LLM. The left approach is QET proposed by hendy2023good and the right part is our proposed PPLT and JDM, which quickly determines when to use LLM based on source sentence.
  • Figure 3: LLM Translation Prompt