Lexicalized Constituency Parsing for Middle Dutch: Low-resource Training and Cross-Domain Generalization

TL;DR

This work tackles constituency parsing for a low-resource historical language, Middle Dutch, by adapting the transformer-based Benepar parser with BERT-based representations. It systematically investigates how auxiliary-language transfer and domain adaptation affect in-domain and cross-domain parsing performance, showing that temporally/geographically closer auxiliary languages yield the largest F1 gains (up to +0.73, with 86.21 F1 on Etstoel) and that zero-shot transfer outperforms the traditional PCFG baseline. Phase I finds that PoS-tag auxiliary tasks provide little benefit, while Phase II demonstrates substantial gains from several historical auxiliary languages; Phase III shows that adversarial feature separation and related strategies can boost cross-domain performance, with a practical threshold of about 200 target-domain examples to achieve reliable improvements (around 74.7 F1). Overall, the study achieves state-of-the-art Middle Dutch parsing and offers generalizable insights into neural constituency parsing for low-resource, heterogeneous historical languages, including guidance for incremental treebank construction and domain adaptation strategies.

Abstract

Recent years have seen growing interest in applying neural networks and contextualized word embeddings to the parsing of historical languages. However, most advances have focused on dependency parsing, while constituency parsing for low-resource historical languages like Middle Dutch has received little attention. In this paper, we adapt a transformer-based constituency parser to Middle Dutch, a highly heterogeneous and low-resource language, and investigate methods to improve both its in-domain and cross-domain performance. We show that joint training with higher-resource auxiliary languages increases F1 scores by up to 0.73, with the greatest gains achieved from languages that are geographically and temporally closer to Middle Dutch. We further evaluate strategies for leveraging newly annotated data from additional domains, finding that fine-tuning and data combination yield comparable improvements, and our neural parser consistently outperforms the currently used PCFG-based parser for Middle Dutch. We further explore feature-separation techniques for domain adaptation and demonstrate that a minimum threshold of approximately 200 examples per domain is needed to effectively enhance cross-domain performance.

Figures (2)

• Figure 1: A constituency tree from Etstoel.
• Figure 2: Model architectures. Enc. stands for encoder and GRL for gradient reversal layer.