A Computational Approach to Language Contact -- A Case Study of Persian

TL;DR

The paper investigates whether a monolingual Persian language model encodes structural traces of historical language contact in its intermediate representations. It applies two analyses—variational usable information probing with $I_{ mathcal{V}}(X \rightarrow Y)$ normalized to $\hat{I}_{\nmathcal{V}} \in [0,1]$ and Language Activation Probability Entropy (LAPE) attribution—to ParsBERT using data from Parallel Universal Dependencies across eight test languages with diverse contact histories. The findings show that universal syntactic categories (UPOS) are largely insensitive to contact, while morphological features such as Case and Gender exhibit strong language-specific effects and distributed cues rather than compact, feature-specific units. This work provides a principled computational framework linking language-contact linguistics and neural representations, with implications for typology-aware NLP and the study of language change using monolingual models.

Abstract

We investigate structural traces of language contact in the intermediate representations of a monolingual language model. Focusing on Persian (Farsi) as a historically contact-rich language, we probe the representations of a Persian-trained model when exposed to languages with varying degrees and types of contact with Persian. Our methodology quantifies the amount of linguistic information encoded in intermediate representations and assesses how this information is distributed across model components for different morphosyntactic features. The results show that universal syntactic information is largely insensitive to historical contact, whereas morphological features such as Case and Gender are strongly shaped by language-specific structure, suggesting that contact effects in monolingual language models are selective and structurally constrained.

Figures (16)

• Figure 1: Normalized variational usable information ($\hat{I}_\mathcal{V}$) for language identification.
• Figure 2: The LAPE scores for each language.
• Figure 3: Normalized variational usable information ($\hat{I}_\mathcal{V}$) for UPOS identification.
• Figure 4: The LAPE scores for UPOS categories.
• Figure 5: Normalized variational usable information ($\hat{I}_\mathcal{V}$) for Case prediction.