Using Correspondence Patterns to Identify Irregular Words in Cognate sets Through Leave-One-Out Validation

TL;DR

This paper tackles the challenge of quantifying regularity in cognate correspondence patterns and automating irregular word detection in historical linguistic datasets. It introduces a regularity measure based on the balanced average recurrence of correspondence patterns, with a formal score $S=\exp\left(\frac{1}{N}\sum_{i=1}^{N}\log r_i\right)$, and a leave-one-out validation pipeline to identify irregular cognate forms. Evaluations on simulated and real LexiBank-based data show overall accuracies around 85% on real data, with higher performance on smaller language samples and substantial dataset-dependent variation. The work promises practical benefits for data curation, improved cognate coding, and more reliable proto-language reconstruction in computer-assisted language comparison.

Abstract

Regular sound correspondences constitute the principal evidence in historical language comparison. Despite the heuristic focus on regularity, it is often more an intuitive judgement than a quantified evaluation, and irregularity is more common than expected from the Neogrammarian model. Given the recent progress of computational methods in historical linguistics and the increased availability of standardized lexical data, we are now able to improve our workflows and provide such a quantitative evaluation. Here, we present the balanced average recurrence of correspondence patterns as a new measure of regularity. We also present a new computational method that uses this measure to identify cognate sets that lack regularity with respect to their correspondence patterns. We validate the method through two experiments, using simulated and real data. In the experiments, we employ leave-one-out validation to measure the regularity of cognate sets in which one word form has been replaced by an irregular one, checking how well our method identifies the forms causing the irregularity. Our method achieves an overall accuracy of 85\% with the datasets based on real data. We also show the benefits of working with subsamples of large datasets and how increasing irregularity in the data influences our results. Reflecting on the broader potential of our new regularity measure and the irregular cognate identification method based on it, we conclude that they could play an important role in improving the quality of existing and future datasets in computer-assisted language comparison.

Figures (5)

• Figure 1: Map of all languages in the sample coloured per dataset.
• Figure 2: Artificial example for three cognate sets across four languages with their aligned segments. The columns are numbered and coloured according to their correspondence patterns. The vowel patterns are not further distinguished since all examples in the text refer to the patterns I-III.
• Figure 3: Overall regularity of the 20 datasets in the sample (y-axis) as indicated by measuring the average recurrence of sites in two ways. The left subplot shows the normalized and log-transformed recurrence of each site (x-axis). The right subplot shows the exponential of the mean of that log-transformed recurrence. This score can be interpreted as the balanced average pattern recurrence of an alignment site within each dataset.
• Figure 4: Accuracy for the leave-one-out validation with simulated regular data (y-axis) and random replacement of phones (x-axis) to simulate different levels of irregularity in comparative wordlists.
• Figure 5: Distribution of experiment results for all datasets, split into the 5-doculects and 10-doculects sample settings. The boxplot presents the 50% distribution of all results.