Leveraging In-Context Learning and Retrieval-Augmented Generation for Automatic Question Generation in Educational Domains

TL;DR

This work targets contextually relevant automatic question generation in education by comparing In-Context Learning (ICL) and Retrieval-Augmented Generation (RAG) and introducing a Hybrid model that integrates both. It employs GPT-4 for ICL with few-shot prompts and BART with FAISS-based retrieval for RAG, evaluating on the EduProbe dataset derived from NCERT materials. Automated metrics favor ICL (especially with larger few-shot sets) and the Hybrid approach, while human evaluators bestow the highest scores on the Hybrid configuration for grammaticality, appropriateness, relevance, and complexity. The findings underscore the value of combining external retrieval with guided few-shot prompts to produce pedagogically richer, contextually aligned questions, with implications for scalable, curriculum-aware AQG systems.

Abstract

Question generation in education is a time-consuming and cognitively demanding task, as it requires creating questions that are both contextually relevant and pedagogically sound. Current automated question generation methods often generate questions that are out of context. In this work, we explore advanced techniques for automated question generation in educational contexts, focusing on In-Context Learning (ICL), Retrieval-Augmented Generation (RAG), and a novel Hybrid Model that merges both methods. We implement GPT-4 for ICL using few-shot examples and BART with a retrieval module for RAG. The Hybrid Model combines RAG and ICL to address these issues and improve question quality. Evaluation is conducted using automated metrics, followed by human evaluation metrics. Our results show that both the ICL approach and the Hybrid Model consistently outperform other methods, including baseline models, by generating more contextually accurate and relevant questions.