HausaNLP at SemEval-2025 Task 2: Entity-Aware Fine-tuning vs. Prompt Engineering in Entity-Aware Machine Translation

TL;DR

This work tackles entity-aware machine translation within SemEval-2025 Task 2 by comparing four approaches: bilingual fine-tuning of the NLLB-200 model with Wikidata-derived named-entity translations and two prompt-based strategies for the closed-source Gemini model (zero-shot and few-shot). Across ten target languages, Gemini generally preserves entity translations better and, in zero-shot form, delivers the strongest overall performance, while few-shot gains are limited for many languages. European languages tend to achieve higher scores than Asian languages, with notable strengths for Spanish and Italian, and Turkish showing strong results under prompting despite data limitations for fine-tuning. The study demonstrates the practical value of prompt-based techniques for entity preservation in MT and highlights language-specific factors and data availability as key determinants of method effectiveness.

Abstract

This paper presents our findings for SemEval 2025 Task 2, a shared task on entity-aware machine translation (EA-MT). The goal of this task is to develop translation models that can accurately translate English sentences into target languages, with a particular focus on handling named entities, which often pose challenges for MT systems. The task covers 10 target languages with English as the source. In this paper, we describe the different systems we employed, detail our results, and discuss insights gained from our experiments.