KpopMT: Translation Dataset with Terminology for Kpop Fandom

TL;DR

The paper addresses the gap in MT for translating social-group terminologies by introducing KpopMT, a terminology-tagged Korean–English dataset built from expert translations of fandom content. It employs a two-phase construction (Sentence Phase and Terminology Phase) to produce 1k parallel sentences annotated with Group-Lexicon, Group-NE, and Slang terms, plus a termbase and fandom monolingual data. Through extensive experiments with open-source and proprietary MT systems, including GPT models, the study reveals overall low translation and terminological accuracy, with GPTs best among the tested methods but still struggling with group-specific terms, particularly Group-Lexicon, and notes that back-translation from fandom data can be noisy. The work provides a publicly available benchmark to drive terminology-aware MT research in social groups and offers insights into domain adaptation challenges and the need for better handling of culturally specific language in translation systems.

Abstract

While machines learn from existing corpora, humans have the unique capability to establish and accept new language systems. This makes human form unique language systems within social groups. Aligning with this, we focus on a gap remaining in addressing translation challenges within social groups, where in-group members utilize unique terminologies. We propose KpopMT dataset, which aims to fill this gap by enabling precise terminology translation, choosing Kpop fandom as an initiative for social groups given its global popularity. Expert translators provide 1k English translations for Korean posts and comments, each annotated with specific terminology within social groups' language systems. We evaluate existing translation systems including GPT models on KpopMT to identify their failure cases. Results show overall low scores, underscoring the challenges of reflecting group-specific terminologies and styles in translation. We make KpopMT publicly available.