Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

McMining: Automated Discovery of Misconceptions in Student Code

Erfan Al-Hossami, Razvan Bunescu

TL;DR

McMining tackles the automated discovery of programming misconceptions from student code, introducing a benchmark dataset and two LLM-based mining tools (McMiner-S and McMiner-M) built atop a misconception-injection tool (McInject). The approach demonstrates that multi-instance mining across code bags yields strong accuracy (best 82.0% on McMiner-M across evaluated LLMs) and can surface novel misconceptions beyond a predefined bank. By enabling the articulation of false beliefs about language concepts, the work supports scalable, personalized feedback in programming education. The dataset and tools pave the way for integrating automated misconception detection into classroom workflows and targeted pedagogical interventions.

Abstract

When learning to code, students often develop misconceptions about various programming language concepts. These can not only lead to bugs or inefficient code, but also slow down the learning of related concepts. In this paper, we introduce McMining, the task of mining programming misconceptions from samples of code from a student. To enable the training and evaluation of McMining systems, we develop an extensible benchmark dataset of misconceptions together with a large set of code samples where these misconceptions are manifested. We then introduce two LLM-based McMiner approaches and through extensive evaluations show that models from the Gemini, Claude, and GPT families are effective at discovering misconceptions in student code.

McMining: Automated Discovery of Misconceptions in Student Code

TL;DR

McMining tackles the automated discovery of programming misconceptions from student code, introducing a benchmark dataset and two LLM-based mining tools (McMiner-S and McMiner-M) built atop a misconception-injection tool (McInject). The approach demonstrates that multi-instance mining across code bags yields strong accuracy (best 82.0% on McMiner-M across evaluated LLMs) and can surface novel misconceptions beyond a predefined bank. By enabling the articulation of false beliefs about language concepts, the work supports scalable, personalized feedback in programming education. The dataset and tools pave the way for integrating automated misconception detection into classroom workflows and targeted pedagogical interventions.

Abstract

When learning to code, students often develop misconceptions about various programming language concepts. These can not only lead to bugs or inefficient code, but also slow down the learning of related concepts. In this paper, we introduce McMining, the task of mining programming misconceptions from samples of code from a student. To enable the training and evaluation of McMining systems, we develop an extensible benchmark dataset of misconceptions together with a large set of code samples where these misconceptions are manifested. We then introduce two LLM-based McMiner approaches and through extensive evaluations show that models from the Gemini, Claude, and GPT families are effective at discovering misconceptions in student code.

Paper Structure

This paper contains 28 sections, 2 equations, 8 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction and Motivation
  2. Related Work
  3. Task Definition and Benchmark Dataset
  4. The McInject Tool
  5. The McMiner Tools
  6. Experimental Evaluations
  7. Conclusion
  8. Limitations
  9. McInject Qualitative Analysis
  10. Success Patterns
  11. Failure Patterns
  12. Ambiguous Correctness
  13. Unnatural Code
  14. McMiner Analysis
  15. Performance on Correct-Only vs. Misconception Bags
  16. ...and 13 more sections

Figures (8)

  • Figure 1: Problem-code pair that exhibits a potential programming misconception about the range function.
  • Figure 2: Web application interface of the McMiner-S tool. Users are able to input a programming problem and a student implementation, and analyze the code for any potential misconceptions.
  • Figure 3: Misconception description writing guidelines.
  • Figure 4: Prompt template for McInject tool.
  • Figure 5: Prompt template for McMiner-S (single-instance mining). The full template includes additional metadata fields and guidelines.
  • ...and 3 more figures