Traditional Readability Formulas Compared for English

TL;DR

Readability assessment in NLP is revisited by recalibrating five traditional English formulas and introducing NERF. The approach leverages a diverse set of datasets, notably CCB for calibration, and LingFeat-based linguistic features, using non-linear least-squares fitting to obtain recalibrated coefficients and a linear NERF model. Across passage- and sentence-level tasks and in medical documents, recalibrated formulas and NERF achieve closer alignment with human judgments and better performance in text simplification; a public Python library enables practical adoption. The work also discusses limitations of surface-based metrics and highlights the importance of lexico-semantic and syntactic features for cross-domain readability estimation.

Abstract

Traditional English readability formulas, or equations, were largely developed in the 20th century. Nonetheless, many researchers still rely on them for various NLP applications. This phenomenon is presumably due to the convenience and straightforwardness of readability formulas. In this work, we contribute to the NLP community by 1. introducing New English Readability Formula (NERF), 2. recalibrating the coefficients of old readability formulas (Flesch-Kincaid Grade Level, Fog Index, SMOG Index, Coleman-Liau Index, and Automated Readability Index), 3. evaluating the readability formulas, for use in text simplification studies and medical texts, and 4. developing a Python-based program for the wide application to various NLP projects.

Figures (1)

• Figure 1: On medical texts. NERF, against five others.