Towards a resource for multilingual lexicons: an MT assisted and human-in-the-loop multilingual parallel corpus with multi-word expression annotation
Lifeng Han, Najet Hadj Mohamed, Malak Rassem, Gareth Jones, Alan Smeaton, Goran Nenadic
TL;DR
AlphaMWE presents a MT-assisted multilingual parallel corpus annotated for verbal MWEs across six languages, built atop the PARSEME vMWE English data and refined through human post-editing and cross-validation. The work documents a rigorous workflow for MT system selection, post-editing, and bilingual annotation, and analyzes MT errors using an extended HOPE metric with MWE-specific categories. Key contributions include a 750-sentence English core with aligned multilingual vMWEs, a detailed error taxonomy for MWEs, and insights from English→Arabic case studies to inform MT evaluation and MWE lexicography. The resource supports MT testing, cross-lingual information extraction, and multi-word term lexicography, while highlighting substantial gaps in current MT systems for idiomatic and metaphorical MWEs. AlphaMWE is publicly released to facilitate broader multilingual NLP research and future expansion to additional languages and MWE types.
Abstract
In this work, we introduce the construction of a machine translation (MT) assisted and human-in-the-loop multilingual parallel corpus with annotations of multi-word expressions (MWEs), named AlphaMWE. The MWEs include verbal MWEs (vMWEs) defined in the PARSEME shared task that have a verb as the head of the studied terms. The annotated vMWEs are also bilingually and multilingually aligned manually. The languages covered include Arabic, Chinese, English, German, Italian, and Polish, of which, the Arabic corpus includes both standard and dialectal variations from Egypt and Tunisia. Our original English corpus is extracted from the PARSEME shared task in 2018. We performed machine translation of this source corpus followed by human post-editing and annotation of target MWEs. Strict quality control was applied for error limitation, i.e., each MT output sentence received first manual post-editing and annotation plus a second manual quality rechecking till annotators' consensus is reached. One of our findings during corpora preparation is that accurate translation of MWEs presents challenges to MT systems, as reflected by the outcomes of human-in-the-loop metric HOPE. To facilitate further MT research, we present a categorisation of the error types encountered by MT systems in performing MWE-related translation. To acquire a broader view of MT issues, we selected four popular state-of-the-art MT systems for comparison, namely Microsoft Bing Translator, GoogleMT, Baidu Fanyi, and DeepL MT. Because of the noise removal, translation post-editing, and MWE annotation by human professionals, we believe the AlphaMWE data set will be an asset for both monolingual and cross-lingual research, such as multi-word term lexicography, MT, and information extraction.