Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Lazy Student's Dream: ChatGPT Passing an Engineering Course on Its Own

Gokul Puthumanaillam, Timothy Bretl, Melkior Ornik

TL;DR

This study systematically evaluates a full undergraduate control systems course (AE 353) using the free GPT-4-based ChatGPT under a minimal-effort student model to determine AI capabilities across 115 deliverables, including MCQ, coding, and open-ended problems. It employs three prompting strategies—image-based, simplified notation, and context-enhanced prompting—with zero-shot and multi-shot variants, and uses standardized, no-edit translation to gradable formats. The results show an overall LLM performance of $82.24\%$ (a B) versus a class average of $84.99\%$, with strongest performance on structured tasks and notable weaknesses on open-ended projects; auto-graded exam components outperform written portions, and programming projects reveal the greatest gaps. The findings highlight the potential for AI-era course design, advocating integrated, open-ended assessments and explicit reasoning in exams to better discern human understanding, while suggesting opportunities to leverage AI as a learning aid rather than merely policing its use.

Abstract

This paper presents a comprehensive investigation into the capability of Large Language Models (LLMs) to successfully complete a semester-long undergraduate control systems course. Through evaluation of 115 course deliverables, we assess LLM performance using ChatGPT under a "minimal effort" protocol that simulates realistic student usage patterns. The investigation employs a rigorous testing methodology across multiple assessment formats, from auto-graded multiple choice questions to complex Python programming tasks and long-form analytical writing. Our analysis provides quantitative insights into AI's strengths and limitations in handling mathematical formulations, coding challenges, and theoretical concepts in control systems engineering. The LLM achieved a B-grade performance (82.24\%), approaching but not exceeding the class average (84.99\%), with strongest results in structured assignments and greatest limitations in open-ended projects. The findings inform discussions about course design adaptation in response to AI advancement, moving beyond simple prohibition towards thoughtful integration of these tools in engineering education. Additional materials including syllabus, examination papers, design projects, and example responses can be found at the project website: https://gradegpt.github.io.

The Lazy Student's Dream: ChatGPT Passing an Engineering Course on Its Own

TL;DR

This study systematically evaluates a full undergraduate control systems course (AE 353) using the free GPT-4-based ChatGPT under a minimal-effort student model to determine AI capabilities across 115 deliverables, including MCQ, coding, and open-ended problems. It employs three prompting strategies—image-based, simplified notation, and context-enhanced prompting—with zero-shot and multi-shot variants, and uses standardized, no-edit translation to gradable formats. The results show an overall LLM performance of (a B) versus a class average of , with strongest performance on structured tasks and notable weaknesses on open-ended projects; auto-graded exam components outperform written portions, and programming projects reveal the greatest gaps. The findings highlight the potential for AI-era course design, advocating integrated, open-ended assessments and explicit reasoning in exams to better discern human understanding, while suggesting opportunities to leverage AI as a learning aid rather than merely policing its use.

Abstract

This paper presents a comprehensive investigation into the capability of Large Language Models (LLMs) to successfully complete a semester-long undergraduate control systems course. Through evaluation of 115 course deliverables, we assess LLM performance using ChatGPT under a "minimal effort" protocol that simulates realistic student usage patterns. The investigation employs a rigorous testing methodology across multiple assessment formats, from auto-graded multiple choice questions to complex Python programming tasks and long-form analytical writing. Our analysis provides quantitative insights into AI's strengths and limitations in handling mathematical formulations, coding challenges, and theoretical concepts in control systems engineering. The LLM achieved a B-grade performance (82.24\%), approaching but not exceeding the class average (84.99\%), with strongest results in structured assignments and greatest limitations in open-ended projects. The findings inform discussions about course design adaptation in response to AI advancement, moving beyond simple prohibition towards thoughtful integration of these tools in engineering education. Additional materials including syllabus, examination papers, design projects, and example responses can be found at the project website: https://gradegpt.github.io.

Paper Structure

This paper contains 15 sections, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Current State of LLMs in Education
  4. Existing Studies on AI in Engineering Education
  5. AE 353: Course Structure and Requirements
  6. Methodology
  7. Protocol Development
  8. LLM Selection and Version
  9. Prompting Approaches
  10. Translation Standards and Evaluation Methodology
  11. Results and Analysis
  12. Performance Analysis
  13. Qualitative Analysis of LLM Responses
  14. Discussion
  15. Conclusion

Figures (6)

  • Figure 1: Distribution of assessment types across AE 353.
  • Figure 2: Example of image-based prompting.
  • Figure 3: Example of simplified notation prompting.
  • Figure 4: Example of context-enhanced prompting: Question presentation preceded by relevant lecture material.
  • Figure 5: Histograms depict the student cohort's score distribution. The LLM's performance is overlaid for context-enhanced prompting (blue, dashed) and simplified text (green, dotted) across different assignment types. Image-based prompting is excluded as projects were not graded using this method.
  • ...and 1 more figures