Modern Sequential Analysis and its Applications to Computerized Adaptive Testing

TL;DR

The paper addresses efficient stopping and adaptive design in computerized mastery testing by embedding sequential generalized likelihood ratio (GLR) statistics into sequential experiments. It extends asymptotic optimality results to sequentially generated data and maximum-length constraints, enabling near-minimal expected sample sizes under prescribed error rates. Using a 3-PL IRT framework, it develops adaptive item selection rules and proves that modified Haybittle-Peto type tests achieve substantial reductions in test length while controlling false positives and negatives. Simulation studies demonstrate practical gains over fixed-length testing and standard TSPRT methods, and the framework accommodates exposure control and content balancing for real CAT deployments.

Abstract

After a brief review of recent advances in sequential analysis involving sequential generalized likelihood ratio tests, we discuss their use in psychometric testing and extend the asymptotic optimality theory of these sequential tests to the case of sequentially generated experiments, of particular interest in computerized adaptive testing. We then show how these methods can be used to design adaptive mastery tests, which are asymptotically optimal and are also shown to provide substantial improvements over currently used sequential and fixed length tests.