Cross-lingual Transfer Learning for Javanese Dependency Parsing

TL;DR

This paper tackles dependency parsing for Javanese, a high-speech but low-resource language, by applying cross-lingual transfer learning (TL) and hierarchical transfer learning (HTL) within an encoder–decoder framework featuring a self-attention encoder and a graph-based biaffine decoder. Leveraging the Universal Dependencies corpus and LangRank-guided source-language selection, the study compares training from scratch, TL, and HTL across multiple embeddings, including fastText and BERT variants. The results show TL and HTL outperform baseline training, with HTL delivering the largest gains (up to about 10% in UAS and LAS) and Indonesian serving as a key intermediary language, while language-embedding choices influence performance. The work contributes the first systematic evaluation of TL and HTL for Javanese parsing, offers insights into source-language and embedding selection, and points to future improvements via architectureic refinements and broader language coverage.

Abstract

While structure learning achieves remarkable performance in high-resource languages, the situation differs for under-represented languages due to the scarcity of annotated data. This study focuses on assessing the efficacy of transfer learning in enhancing dependency parsing for Javanese, a language spoken by 80 million individuals but characterized by limited representation in natural language processing. We utilized the Universal Dependencies dataset consisting of dependency treebanks from more than 100 languages, including Javanese. We propose two learning strategies to train the model: transfer learning (TL) and hierarchical transfer learning (HTL). While TL only uses a source language to pre-train the model, the HTL method uses a source language and an intermediate language in the learning process. The results show that our best model uses the HTL method, which improves performance with an increase of 10% for both UAS and LAS evaluations compared to the baseline model.

Figures (4)

• Figure 1: Illustration of standard transfer learning method
• Figure 2: Illustration of hierarchical transfer learning method
• Figure 3: Comparison of the best model evaluation for each scenario
• Figure 4: Comparison of the best model evaluation for each word embedding