Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EduAdapt: A Question Answer Benchmark Dataset for Evaluating Grade-Level Adaptability in LLMs

Numaan Naeem, Abdellah El Mekki, Muhammad Abdul-Mageed

TL;DR

EduAdapt tackles the critical gap of grade-level adaptability in LLMs for K-12 education by introducing a large-scale, NGSS-aligned QA dataset generated from Wikipedia via a two-stage pipeline and validated with human review. The dataset comprises roughly 47.7k QA pairs across nine subjects and four grade bands, with train/dev/test splits, enabling both adaptation and evaluation of LLMs across the full K-12 spectrum. Evaluations on open-source LLMs reveal a persistent gap for early-grade content (Grades 1–5) despite model size, highlighting the need for grade-aware training and prompting strategies. The work provides a publicly available educational QA resource and an evaluation framework to spur development of developmentally aligned educational AI, with plans for multimodal, multilingual, and curriculum-aware extensions.

Abstract

Large language models (LLMs) are transforming education by answering questions, explaining complex concepts, and generating content across a wide range of subjects. Despite strong performance on academic benchmarks, they often fail to tailor responses to students' grade levels. This is a critical need in K-12 education, where age-appropriate vocabulary and explanation are essential for effective learning. Existing models frequently produce outputs that are too advanced or vague for younger learners, and there are no standardized benchmarks to evaluate their ability to adjust across cognitive and developmental stages. To address this gap, we introduce EduAdapt, a benchmark of nearly 48k grade-labeled QA pairs across nine science subjects, spanning Grades 1-12 and grouped into four grade levels. We evaluate a diverse set of open-source LLMs on EduAdapt and find that while larger models generally perform better, they still struggle with generating suitable responses for early-grade students (Grades 1-5). Our work presents the first dataset and evaluation framework for assessing grade-level adaptability in LLMs, aiming to foster more developmentally aligned educational AI systems through better training and prompting strategies. EduAdapt code and datasets are publicly available at https://github.com/NaumanNaeem/EduAdapt.

EduAdapt: A Question Answer Benchmark Dataset for Evaluating Grade-Level Adaptability in LLMs

TL;DR

EduAdapt tackles the critical gap of grade-level adaptability in LLMs for K-12 education by introducing a large-scale, NGSS-aligned QA dataset generated from Wikipedia via a two-stage pipeline and validated with human review. The dataset comprises roughly 47.7k QA pairs across nine subjects and four grade bands, with train/dev/test splits, enabling both adaptation and evaluation of LLMs across the full K-12 spectrum. Evaluations on open-source LLMs reveal a persistent gap for early-grade content (Grades 1–5) despite model size, highlighting the need for grade-aware training and prompting strategies. The work provides a publicly available educational QA resource and an evaluation framework to spur development of developmentally aligned educational AI, with plans for multimodal, multilingual, and curriculum-aware extensions.

Abstract

Large language models (LLMs) are transforming education by answering questions, explaining complex concepts, and generating content across a wide range of subjects. Despite strong performance on academic benchmarks, they often fail to tailor responses to students' grade levels. This is a critical need in K-12 education, where age-appropriate vocabulary and explanation are essential for effective learning. Existing models frequently produce outputs that are too advanced or vague for younger learners, and there are no standardized benchmarks to evaluate their ability to adjust across cognitive and developmental stages. To address this gap, we introduce EduAdapt, a benchmark of nearly 48k grade-labeled QA pairs across nine science subjects, spanning Grades 1-12 and grouped into four grade levels. We evaluate a diverse set of open-source LLMs on EduAdapt and find that while larger models generally perform better, they still struggle with generating suitable responses for early-grade students (Grades 1-5). Our work presents the first dataset and evaluation framework for assessing grade-level adaptability in LLMs, aiming to foster more developmentally aligned educational AI systems through better training and prompting strategies. EduAdapt code and datasets are publicly available at https://github.com/NaumanNaeem/EduAdapt.
Paper Structure (41 sections, 2 figures, 64 tables)

This paper contains 41 sections, 2 figures, 64 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Stage 1: Generation Process
  4. Content Collection from Wikipedia
  5. Question-Answer Generation
  6. Self-Reflection for Quality Assessment
  7. Error Analysis of Low-Quality QA Pairs.
  8. Stage 2: Human Verification Process
  9. Experiments
  10. Evaluated Models.
  11. Evaluated Method.
  12. Performance Metrics.
  13. Implementation Details.
  14. Results and Analysis
  15. Results on MCQs set.
  16. ...and 26 more sections

Figures (2)

  • Figure 1: Overview of the whole pipeline. Phase I covers the EduAdapt creation process, which includes Generation and Human Verification, where Wikipedia articles are transformed into QA pairs and manually validated for quality. Phase II covers LLM Evaluation, where open-source LLMs are tested on EduAdapt's Test set to assess grade-level adaptability.
  • Figure 2: QA Generator: A pipeline that generates QA pairs. Wikipedia passages are first fed into the QA generator to produce QA pairs. Each pair is then evaluated by a self-reflection module, which scores it across five defined criteria. Filtering is applied to retain only those pairs with ratings above 8 on all criteria, resulting in a high-quality QA dataset.