NLP Cluster Analysis of Common Core State Standards and NAEP Item Specifications

TL;DR

This study tests whether empirical clusters derived from embedding-based semantic representations can recover the nominal domain and strand classifications used in CCSS and NAEP item specifications. By processing text with embeddings, reducing dimensionality via principal components, and applying a five-cluster k-means algorithm, the authors compare empirical groupings to nominal classifications and quantify cross-classification accuracy. They report substantial alignment (eighty-two percent for CCSS; ninety-one percent for NAEP) with a handful of mismatches explained by semantic overlap in measurement terminology, suggesting both the usefulness and limits of NLP-driven mapping for standards work. Overall, the approach demonstrates that NLP can efficiently illuminate the structure of educational standards and item specifications, guiding refinement of construct boundaries and informing policy-relevant standard-setting and assessment design.

Abstract

Camilli (2024) proposed a methodology using natural language processing (NLP) to map the relationship of a set of content standards to item specifications. This study provided evidence that NLP can be used to improve the mapping process. As part of this investigation, the nominal classifications of standards and items specifications were used to examine construct equivalence. In the current paper, we determine the strength of empirical support for the semantic distinctiveness of these classifications, which are known as "domains" for Common Core standards, and "strands" for National Assessment of Educational Progress (NAEP) item specifications. This is accomplished by separate k-means clustering for standards and specifications of their corresponding embedding vectors. We then briefly illustrate an application of these findings.