Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accurate Predictions in Education with Discrete Variational Inference

Tom Quilter, Anastasia Ilick, Karen Poon, Richard Turner

TL;DR

The paper tackles the challenge of predicting student success on unseen questions to enable scalable AI tutoring and reduce inequality. It builds on item response theory by evaluating an ability–difficulty model and extended class-interaction variants, and introduces a discrete variational inference framework to handle uncertainty in binary responses, validated on a large open GCSE mathematics dataset. The key findings show that a single overall ability parameter often dominates predictive accuracy, topic-level skill differences offer limited additional predictive power, and discrete variational inference yields notable gains in data-sparse settings, with class-level information providing modest improvements. The work provides practical guidance for adaptive learning systems and contributes a large, openly available benchmark for future research in educational data mining and probabilistic modelling in education.

Abstract

One of the largest drivers of social inequality is unequal access to personal tutoring, with wealthier individuals able to afford it, while the majority cannot. Affordable, effective AI tutors offer a scalable solution. We focus on adaptive learning, predicting whether a student will answer a question correctly, a key component of any effective tutoring system. Yet many platforms struggle to achieve high prediction accuracy, especially in data-sparse settings. To address this, we release the largest open dataset of professionally marked formal mathematics exam responses to date. We introduce a probabilistic modelling framework rooted in Item Response Theory (IRT) that achieves over 80 percent accuracy, setting a new benchmark for mathematics prediction accuracy of formal exam papers. Extending this, our collaborative filtering models incorporate topic-level skill profiles, but reveal a surprising and educationally significant finding, a single latent ability parameter alone is needed to achieve the maximum predictive accuracy. Our main contribution though is deriving and implementing a novel discrete variational inference framework, achieving our highest prediction accuracy in low-data settings and outperforming all classical IRT and matrix factorisation baselines.

Accurate Predictions in Education with Discrete Variational Inference

TL;DR

The paper tackles the challenge of predicting student success on unseen questions to enable scalable AI tutoring and reduce inequality. It builds on item response theory by evaluating an ability–difficulty model and extended class-interaction variants, and introduces a discrete variational inference framework to handle uncertainty in binary responses, validated on a large open GCSE mathematics dataset. The key findings show that a single overall ability parameter often dominates predictive accuracy, topic-level skill differences offer limited additional predictive power, and discrete variational inference yields notable gains in data-sparse settings, with class-level information providing modest improvements. The work provides practical guidance for adaptive learning systems and contributes a large, openly available benchmark for future research in educational data mining and probabilistic modelling in education.

Abstract

One of the largest drivers of social inequality is unequal access to personal tutoring, with wealthier individuals able to afford it, while the majority cannot. Affordable, effective AI tutors offer a scalable solution. We focus on adaptive learning, predicting whether a student will answer a question correctly, a key component of any effective tutoring system. Yet many platforms struggle to achieve high prediction accuracy, especially in data-sparse settings. To address this, we release the largest open dataset of professionally marked formal mathematics exam responses to date. We introduce a probabilistic modelling framework rooted in Item Response Theory (IRT) that achieves over 80 percent accuracy, setting a new benchmark for mathematics prediction accuracy of formal exam papers. Extending this, our collaborative filtering models incorporate topic-level skill profiles, but reveal a surprising and educationally significant finding, a single latent ability parameter alone is needed to achieve the maximum predictive accuracy. Our main contribution though is deriving and implementing a novel discrete variational inference framework, achieving our highest prediction accuracy in low-data settings and outperforming all classical IRT and matrix factorisation baselines.

Paper Structure

This paper contains 23 sections, 21 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Item Response Theory (IRT) in Education.
  4. Collaborative Filtering and Matrix Factorisation.
  5. Knowledge Tracing and Sequential Models.
  6. Positioning of Our Work
  7. The Data
  8. The Model
  9. Ability–Difficulty Model (IRT)
  10. The Interaction Model
  11. The Class Interaction Model
  12. Variational Inference
  13. Results and Discussion
  14. Smaller Datasets Results: Discrete Variational Inference improves accuracy
  15. Interpretability: Why does the class interaction model improve accuracy?
  16. ...and 8 more sections

Figures (4)

  • Figure 1: Illustration of raw data
  • Figure 2: Illustration of preprocessed data
  • Figure 3: Normalised cosine similarity of question vectors $b_{qd}$. The lower left rectangle shows the model has discovered an underlying factor representing geometry, the other Algebra. The red circle shows the model has understood that only high ability classes actually get taught the hardest topics.
  • Figure 4: Prediction accuracy of student averaged across all new students in data pool plotted against the number of questions per student so far revealed to the model