A Novel Approach to Scalable and Automatic Topic-Controlled Question Generation in Education

TL;DR

This work tackles the high teacher workload in education by introducing Topic-Controlled Question Generation (T-CQG) using a fine-tuned T5-small model. It develops novel datasets (SQuAD+, MixSQuAD, MixSQuAD2X, MixKhanQ) and a WikiSemRel-based evaluation to ensure topic alignment within paragraph contexts, while exploring pre-training, data augmentation, and quantisation for scalability. The study demonstrates that TopicQG variants generally improve topical relevance over a Baseline, with data augmentation (TopicQG2X) delivering the strongest topic alignment, and quantisation enabling on-device deployment with minimal performance loss. The approach offers a scalable, privacy-preserving alternative to large proprietary LLMs, potentially reducing teacher workload and enabling personalized education through LMS/ITS integrations. Limitations include limited human evaluation power and the need to explore additional factors like linguistic complexity and generation length in future work.

Abstract

The development of Automatic Question Generation (QG) models has the potential to significantly improve educational practices by reducing the teacher workload associated with creating educational content. This paper introduces a novel approach to educational question generation that controls the topical focus of questions. The proposed Topic-Controlled Question Generation (T-CQG) method enhances the relevance and effectiveness of the generated content for educational purposes. Our approach uses fine-tuning on a pre-trained T5-small model, employing specially created datasets tailored to educational needs. The research further explores the impacts of pre-training strategies, quantisation, and data augmentation on the model's performance. We specifically address the challenge of generating semantically aligned questions with paragraph-level contexts, thereby improving the topic specificity of the generated questions. In addition, we introduce and explore novel evaluation methods to assess the topical relatedness of the generated questions. Our results, validated through rigorous offline and human-backed evaluations, demonstrate that the proposed models effectively generate high-quality, topic-focused questions. These models have the potential to reduce teacher workload and support personalised tutoring systems by serving as bespoke question generators. With its relatively small number of parameters, the proposals not only advance the capabilities of question generation models for handling specific educational topics but also offer a scalable solution that reduces infrastructure costs. This scalability makes them feasible for widespread use in education without reliance on proprietary large language models like ChatGPT.

Figures (2)

• Figure 1: Methodology for generating the different training datasets proposed in the model from the contexts $c$, topics $t$ and target questions $q_t$ (shaded as label) from the SQuAD dataset is illustrated using two random examples from the dataset, example $i$ (green) and example $j$ (pink). The contexts $c_i$ and $c_j$ glued together are concatenated texts treated as a single field in the dataset. The orange rectangles indicate the scope of the datasets while (*) marks the newly proposed datasets.
• Figure 2: Methodology for training and evaluating the Baseline model (black), TopicQGedu model (green, RQ3), TopicQG model (dark red, RQ2), its post-training quantised counterparts, TopicQG8bit model(medium red, RQ4), TopicQG4bit model(light red, RQ4) and TopicQG2X model (blue, RQ5). The numbered circles indicate different experimental pathways that test different research questions. The shaded grey box indicates that the T5-Small model was available pre-trained prior to the experiments while the non-shaded models contained parameters trained during the experiments.