PEOAT: Personalization-Guided Evolutionary Question Assembly for One-Shot Adaptive Testing

TL;DR

This work introduces One-Shot Adaptive Testing (OAT) to overcome the sequential interaction costs of traditional CAT by selecting a fixed-length set of items per learner. It presents PEOAT, a personalization-guided evolutionary framework with three components: personalization-aware initialization, cognitive-enhanced evolutionary search (schema-preserving crossover and information-guided mutation), and diversity-preserving environmental selection. Through extensive experiments on two real-world datasets, PEOAT consistently outperforms eight CAT baselines, with especially strong gains at short test lengths, demonstrating its potential for efficient, personalized assessment in large-scale or resource-constrained settings. The study also provides ablations and case studies that highlight the importance of personalized search space design and cognitive signals in guiding effective item assembly.

Abstract

With the rapid advancement of intelligent education, Computerized Adaptive Testing (CAT) has attracted increasing attention by integrating educational psychology with deep learning technologies. Unlike traditional paper-and-pencil testing, CAT aims to efficiently and accurately assess examinee abilities by adaptively selecting the most suitable items during the assessment process. However, its real-time and sequential nature presents limitations in practical scenarios, particularly in large-scale assessments where interaction costs are high, or in sensitive domains such as psychological evaluations where minimizing noise and interference is essential. These challenges constrain the applicability of conventional CAT methods in time-sensitive or resourceconstrained environments. To this end, we first introduce a novel task called one-shot adaptive testing (OAT), which aims to select a fixed set of optimal items for each test-taker in a one-time selection. Meanwhile, we propose PEOAT, a Personalization-guided Evolutionary question assembly framework for One-shot Adaptive Testing from the perspective of combinatorial optimization. Specifically, we began by designing a personalization-aware initialization strategy that integrates differences between examinee ability and exercise difficulty, using multi-strategy sampling to construct a diverse and informative initial population. Building on this, we proposed a cognitive-enhanced evolutionary framework incorporating schema-preserving crossover and cognitively guided mutation to enable efficient exploration through informative signals. To maintain diversity without compromising fitness, we further introduced a diversity-aware environmental selection mechanism. The effectiveness of PEOAT is validated through extensive experiments on two datasets, complemented by case studies that uncovered valuable insights.

Figures (6)

• Figure 1: (a) The process of computerized adaptive testing; (b) Comparison of two adaptive testing tasks (CAT & OAT).
• Figure 2: The overview architecture of our proposed PEOAT model. (a) The personalization-aware population initialization. (b) The the cognitive-enhanced evolutionary search. (c) The diversity-preserving environmental selection. Best viewed in color.
• Figure 3: Performance of ablation studies conducted on the JUNYI, where “w/o” means removing the target module.
• Figure 4: Sensitivity analysis of the distance threshold $\tau$ of the environmental selection on the JUNYI dataset.
• Figure 5: Case study of the performance evolution of the assembled question populations on the JUNYI dataset.