Examining GPT's Capability to Generate and Map Course Concepts and Their Relationship

TL;DR

This study investigates the feasibility of using GPT-family LLMs to automatically generate course concepts and identify inter-conceptual relations from MOOC data, addressing the labor-intensive nature of manual concept extraction. It designs a structured prompt-based workflow with two concept-level tasks (Concept Generation, Concept Extraction) and one relation-level task (Relation Identification), evaluating them under six information-granularity prompts and comparing against classical NLP baselines. Using the XuetangX MOOC dataset ($683$ courses, $25{,}161$ concepts, $1{,}027$ prerequisite relations) and human expert judgments, the results show GPTs can generate high-quality, even low-frequency concepts beyond explicit text, and can predict subtle prerequisite relations, outperforming baselines in many settings. The findings suggest GPT-based approaches can meaningfully support educational content selection and delivery, with further work including broader user studies and cross-domain evaluation of additional LLMs.

Abstract

Extracting key concepts and their relationships from course information and materials facilitates the provision of visualizations and recommendations for learners who need to select the right courses to take from a large number of courses. However, identifying and extracting themes manually is labor-intensive and time-consuming. Previous machine learning-based methods to extract relevant concepts from courses heavily rely on detailed course materials, which necessitates labor-intensive preparation of course materials. This paper investigates the potential of LLMs such as GPT in automatically generating course concepts and their relations. Specifically, we design a suite of prompts and provide GPT with the course information with different levels of detail, thereby generating high-quality course concepts and identifying their relations. Furthermore, we comprehensively evaluate the quality of the generated concepts and relationships through extensive experiments. Our results demonstrate the viability of LLMs as a tool for supporting educational content selection and delivery.

Figures (7)

• Figure 1: Workflow of utilizing GPT to perform two concept-level tasks and one relation-level task. (a) and (b) are concept-level tasks, while (c) is relation-level task.
• Figure 2: Examples of concept generation and extraction task prompts.
• Figure 3: Examples of relation identification task prompts.
• Figure 4: Performance comparison of different GPT models and prompts.
• Figure 5: Human evaluation of the science and non-science courses, along with examples of concept generation in both fields.