Using Letter Positional Probabilities to Assess Word Complexity

TL;DR

This study treats word complexity as a latent construct (LC) and demonstrates that low-level orthographic structure, captured via letter positional probabilities (LPPs) from one-hot encoded words, reliably distinguishes simple from complex word sets. By constructing multiple datasets from children’s books and academic corpora, the authors identify a core set of significant LPPs (66 variables common across experiments) in the first six letter positions and build predictive classifiers that achieve high accuracy (up to 97% at extremes) in ranking words by LC. Across three experiments and an application to a large dictionary, the work shows that LC can be inferred directly from word form, with cross-dataset generalization (70% accuracy on a new dataset using the common LPPs) and strong alignment with human-intuition proxies, while avoiding subjective rating biases. The approach provides a scalable, data-driven metric for word complexity that can support text simplification, educational resource design, and ESL assessment, and it offers concrete, explainable features (specific letter-position probabilities) for fine-grained word ranking.

Abstract

Word complexity is defined in a number of different ways. Psycholinguistic, morphological and lexical proxies are often used. Human ratings are also used. The problem here is that these proxies do not measure complexity directly, and human ratings are susceptible to subjective bias. In this study we contend that some form of 'latent complexity' can be approximated by using samples of simple and complex words. We use a sample of 'simple' words from primary school picture books and a sample of 'complex' words from high school and academic settings. In order to analyse the differences between these classes, we look at the letter positional probabilities (LPPs). We find strong statistical associations between several LPPs and complexity. For example, simple words are significantly (p<.001) more likely to start with w, b, s, h, g, k, j, t, y or f, while complex words are significantly (p<.001) more likely to start with i, a, e, r, v, u or d. We find similar strong associations for subsequent letter positions, with 84 letter-position variables in the first 6 positions being significant at the p<.001 level. We then use LPPs as variables in creating a classifier which can classify the two classes with an 83% accuracy. We test these findings using a second data set, with 66 LPPs significant (p<.001) in the first 6 positions common to both datasets. We use these 66 variables to create a classifier that is able to classify a third dataset with an accuracy of 70%. Finally, we create a fourth sample by combining the extreme high and low scoring words generated by three classifiers built on the first three separate datasets and use this sample to build a classifier which has an accuracy of 97%. We use this to score the four levels of English word groups from an ESL program.