Domain-Specific Quality Estimation for Machine Translation in Low-Resource Scenarios

TL;DR

AlOPE, a framework for LLM-based QE that uses Low-Rank Adaptation with regression heads attached to selected intermediate Transformer layers is adopted, and results show that intermediate-layer adaptation consistently improves QE performance, with gains in semantically complex domains, indicating a path toward more robust QE in practical scenarios.

Abstract

Quality Estimation (QE) is essential for assessing machine translation quality in reference-less settings, particularly for domain-specific and low-resource language scenarios. In this paper, we investigate sentence-level QE for English to Indic machine translation across four domains (Healthcare, Legal, Tourism, and General) and five language pairs. We systematically compare zero-shot, few-shot, and guideline-anchored prompting across selected closed-weight and open-weight LLMs. Findings indicate that while closed-weight models achieve strong performance via prompting alone, prompt-only approaches remain fragile for open-weight models, especially in high-risk domains. To address this, we adopt ALOPE, a framework for LLM-based QE that uses Low-Rank Adaptation with regression heads attached to selected intermediate Transformer layers. We also extend ALOPE with recently proposed Low-Rank Multiplicative Adaptation (LoRMA). Our results show that intermediate-layer adaptation consistently improves QE performance, with gains in semantically complex domains, indicating a path toward more robust QE in practical scenarios. We release code and domain-specific QE datasets publicly to support further research.

Figures (7)

• Figure 1: Methodological Framework uses open-weight models for (i) prompt-only approaches (zero-shot, few-shot, guideline-anchored), and (ii) ALOPE adaptation with LoRA/LoRMA.
• Figure 2: Few-shot QE Prompt (Without Guidelines)
• Figure 3: Few-shot QE Prompt (With Guidelines)
• Figure 4: Zero-shot prompt
• Figure 5: ALOPE prompt