R2T: Rule-Encoded Loss Functions for Low-Resource Sequence Tagging

TL;DR

R2T tackles low-resource sequence tagging by integrating a multi-tiered linguistic rule system directly into the neural training objective, coupled with an adaptive OOV loss to express principled uncertainty. The approach blends a neural architecture (BiLSTM or Transformer) with four rule tiers and a four-term loss, enabling unsupervised POS tagging that outperforms small supervised baselines and serving as data-efficient pre-training for NER in minimal-label regimes. On Zarma, R2T-BiLSTM achieves near-supervised performance without labeled data, while a rule-informed Transformer can reach strong performance after limited supervision; the framework also demonstrates cross-language applicability (Bambara) and motivates a broader Principled Learning paradigm. To support future work, the authors release ZarmaPOS-Bench and ZarmaNER-600, providing substantial silver- and gold-standard resources to catalyze research in low-resource NLP.

Abstract

We introduce the Rule-to-Tag (R2T) framework, a hybrid approach that integrates a multi-tiered system of linguistic rules directly into a neural network's training objective. R2T's novelty lies in its adaptive loss function, which includes a regularization term that teaches the model to handle out-of-vocabulary (OOV) words with principled uncertainty. We frame this work as a case study in a paradigm we call principled learning (PrL), where models are trained with explicit task constraints rather than on labeled examples alone. Our experiments on Zarma part-of-speech (POS) tagging show that the R2T-BiLSTM model, trained only on unlabeled text, achieves 98.2% accuracy, outperforming baselines like AfriBERTa fine-tuned on 300 labeled sentences. We further show that for more complex tasks like named entity recognition (NER), R2T serves as a powerful pre-training step; a model pre-trained with R2T and fine-tuned on just 50 labeled sentences outperformes a baseline trained on 300.

Figures (6)

• Figure 1: Pipeline view of R2T. R2T has two parts: unsupervised training guided by rule-tier losses, and optional supervised fine-tuning (R2T-SFT)
• Figure 2: The R2T framework. A multi-tiered rule system is translated into distinct loss components that guide the training of a neural sequence tagger. The lexical and syntactic losses enforce known grammar, while the distributional and adaptive OOV losses regularize the model's predictions, ensuring robustness and principled handling of uncertainty.
• Figure 3: Architecture of the R2T-BiLSTM model.
• Figure 4: Architecture of the R2T-Transformer model.
• Figure 5: Data-efficiency comparison for Zarma NER. Note: The points for AfriBERTa at 50 examples and R2T at 300 examples are interpolated/projected to illustrate the learning trajectories.