Large Language Models in Computer Science Education: A Systematic Literature Review

TL;DR

A comprehensive systematic literature review is presented to examine the impact of LLMs in computer science and computer engineering education, and analyzes their effectiveness in enhancing the learning experience, supporting personalized education, and aiding educators in curriculum development.

Abstract

Large language models (LLMs) are becoming increasingly better at a wide range of Natural Language Processing tasks (NLP), such as text generation and understanding. Recently, these models have extended their capabilities to coding tasks, bridging the gap between natural languages (NL) and programming languages (PL). Foundational models such as the Generative Pre-trained Transformer (GPT) and LLaMA series have set strong baseline performances in various NL and PL tasks. Additionally, several models have been fine-tuned specifically for code generation, showing significant improvements in code-related applications. Both foundational and fine-tuned models are increasingly used in education, helping students write, debug, and understand code. We present a comprehensive systematic literature review to examine the impact of LLMs in computer science and computer engineering education. We analyze their effectiveness in enhancing the learning experience, supporting personalized education, and aiding educators in curriculum development. We address five research questions to uncover insights into how LLMs contribute to educational outcomes, identify challenges, and suggest directions for future research.

Figures (3)

• Figure 1: Educational Levels where LLMs are used.
• Figure 2: Top 10 Programming Languages.
• Figure 3: Most commonly used LLMs. Others include models that are used only once. A work may use more than one LLM.