Survey of Computerized Adaptive Testing: A Machine Learning Perspective

TL;DR

This survey analyzes how machine learning enriches Computerized Adaptive Testing (CAT) by examining four life-cycle components: Cognitive Diagnosis Models for proficiency estimation, question selection algorithms, question bank construction, and test control. It contrasts traditional statistical methods (e.g., Fisher and KL information) with modern data-driven approaches (reinforcement learning, meta-learning, active learning) and discusses model-agnostic strategies, robustness, fairness, and search efficiency. The paper highlights practical evaluation via simulations and real datasets, and it points to future directions including multi-dimensional assessment, MST, generative AI integration, explainability, and AI-system evaluation. By bridging psychometrics and ML, it advocates an interdisciplinary framework for scalable, fair, and explainable CAT systems with open-source tooling (EduCAT).

Abstract

Computerized Adaptive Testing (CAT) provides an efficient and tailored method for assessing the proficiency of examinees, by dynamically adjusting test questions based on their performance. Widely adopted across diverse fields like education, healthcare, sports, and sociology, CAT has revolutionized testing practices. While traditional methods rely on psychometrics and statistics, the increasing complexity of large-scale testing has spurred the integration of machine learning techniques. This paper aims to provide a machine learning-focused survey on CAT, presenting a fresh perspective on this adaptive testing method. By examining the test question selection algorithm at the heart of CAT's adaptivity, we shed light on its functionality. Furthermore, we delve into cognitive diagnosis models, question bank construction, and test control within CAT, exploring how machine learning can optimize these components. Through an analysis of current methods, strengths, limitations, and challenges, we strive to develop robust, fair, and efficient CAT systems. By bridging psychometric-driven CAT research with machine learning, this survey advocates for a more inclusive and interdisciplinary approach to the future of adaptive testing.

Figures (10)

• Figure 1: If (a) traditional paper-and-pencil test is "one-for-all", then (b) CAT is "one-for-each". Each examinee gets a personalized test that adapts to his/her proficiency level and knowledge, ensuring each question accurately assesses and challenges the examinee
• Figure 2: The workflow of CAT: At step $t$, the selection algorithm adaptively selects next question $q_{t+1}$ based on examinee's current proficiency ${\theta}^{t}$ estimated by CDM.
• Figure 3: Summary of Computerized Adaptive Testing methods in machine learning perspective.
• Figure 4: Illustration of KL and Fisher information functions for two questions (Question 1: $\alpha=1.7, \beta=1.9, c=0.1$; Question 2: $\alpha=2.3, \beta=0.5, c=0.3$). Assuming the current proficiency estimate $\hat{\theta}^t=1$. The KL information (left) for the given question represents an integral centered around $\hat{\theta}^t$, while the Fisher information (right) corresponds to the value at the specific point $\hat{\theta}^t$.
• Figure 5: Top: The Active Learning Framework. Bottom: The relationship and correspondence between each component of Active Learning and those of CAT.

Theorems & Definitions (2)

• Definition 1: Definition of CAT
• Theorem 1: The asymptotic distribution of MLE proficiency estimate ross2014first