Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Taiwanese Hokkien Dual Translation by Exploring and Standardizing of Four Writing Systems

Bo-Han Lu, Yi-Hsuan Lin, En-Shiun Annie Lee, Richard Tzong-Han Tsai

TL;DR

This work tackles translation for a low-resource language, Taiwanese Hokkien, by building a dual translation system with Traditional Mandarin Chinese and English. It leverages a Mandarin-specialized LLaMA-2/TAIDE-7B foundation, augments vocabulary, and conducts extensive continued pre-training on multi-script Hokkien data, followed by instruction-based fine-tuning. A key finding is that pre-training on all available Hokkien data and standardizing scripts into HAN can improve cross-script translations (HAN↔ZH/EN), while vocabulary expansion yields mixed results. The authors also introduce a GPT-4-based evaluation approach with back-translation to reliably assess translation quality for LR languages, and plan to release the translation tools to support broader data generation and research in Taiwanese Hokkien NLP.

Abstract

Machine translation focuses mainly on high-resource languages (HRLs), while low-resource languages (LRLs) like Taiwanese Hokkien are relatively under-explored. The study aims to address this gap by developing a dual translation model between Taiwanese Hokkien and both Traditional Mandarin Chinese and English. We employ a pre-trained LLaMA 2-7B model specialized in Traditional Mandarin Chinese to leverage the orthographic similarities between Taiwanese Hokkien Han and Traditional Mandarin Chinese. Our comprehensive experiments involve translation tasks across various writing systems of Taiwanese Hokkien as well as between Taiwanese Hokkien and other HRLs. We find that the use of a limited monolingual corpus still further improves the model's Taiwanese Hokkien capabilities. We then utilize our translation model to standardize all Taiwanese Hokkien writing systems into Hokkien Han, resulting in further performance improvements. Additionally, we introduce an evaluation method incorporating back-translation and GPT-4 to ensure reliable translation quality assessment even for LRLs. The study contributes to narrowing the resource gap for Taiwanese Hokkien and empirically investigates the advantages and limitations of pre-training and fine-tuning based on LLaMA 2.

Enhancing Taiwanese Hokkien Dual Translation by Exploring and Standardizing of Four Writing Systems

TL;DR

This work tackles translation for a low-resource language, Taiwanese Hokkien, by building a dual translation system with Traditional Mandarin Chinese and English. It leverages a Mandarin-specialized LLaMA-2/TAIDE-7B foundation, augments vocabulary, and conducts extensive continued pre-training on multi-script Hokkien data, followed by instruction-based fine-tuning. A key finding is that pre-training on all available Hokkien data and standardizing scripts into HAN can improve cross-script translations (HAN↔ZH/EN), while vocabulary expansion yields mixed results. The authors also introduce a GPT-4-based evaluation approach with back-translation to reliably assess translation quality for LR languages, and plan to release the translation tools to support broader data generation and research in Taiwanese Hokkien NLP.

Abstract

Machine translation focuses mainly on high-resource languages (HRLs), while low-resource languages (LRLs) like Taiwanese Hokkien are relatively under-explored. The study aims to address this gap by developing a dual translation model between Taiwanese Hokkien and both Traditional Mandarin Chinese and English. We employ a pre-trained LLaMA 2-7B model specialized in Traditional Mandarin Chinese to leverage the orthographic similarities between Taiwanese Hokkien Han and Traditional Mandarin Chinese. Our comprehensive experiments involve translation tasks across various writing systems of Taiwanese Hokkien as well as between Taiwanese Hokkien and other HRLs. We find that the use of a limited monolingual corpus still further improves the model's Taiwanese Hokkien capabilities. We then utilize our translation model to standardize all Taiwanese Hokkien writing systems into Hokkien Han, resulting in further performance improvements. Additionally, we introduce an evaluation method incorporating back-translation and GPT-4 to ensure reliable translation quality assessment even for LRLs. The study contributes to narrowing the resource gap for Taiwanese Hokkien and empirically investigates the advantages and limitations of pre-training and fine-tuning based on LLaMA 2.
Paper Structure (37 sections, 4 figures, 10 tables)

This paper contains 37 sections, 4 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Writing System Diversity in Hokkien
  4. Semantic Divergence of Shared Chinese Characters in HAN and ZH
  5. Related Work
  6. Large Language Models in Translation
  7. Neural Machine Translation in Hokkien
  8. Methodology
  9. Corpus Preparation
  10. Monolingual Datasets
  11. Parallel Datasets
  12. Model Training
  13. Pre-trained Large Language Model
  14. Vocabulary Extension
  15. Continued Pre-training
  16. ...and 22 more sections

Figures (4)

  • Figure 1: The flowchart of data standardization used to create an advanced Taiwanese Hokkien Han dual translator (HAN-ZH and HAN-EN). The dataset in TL was converted to POJ using an one-to-one mapping rule, allowing for a consistent representation of the Hokkien phonetic sounds.
  • Figure 2: Data distribution of monolingual corpora for continued pre-training
  • Figure 3: Continued pre-training corpora JSD with dendrogram
  • Figure 4: Fine-tuning data JSD