Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Large Language Models for Automated Homework Assessment in Undergraduate Circuit Analysis

Liangliang Chen, Huiru Xie, Zhihao Qin, Yiming Guo, Jacqueline Rohde, Ying Zhang

TL;DR

This work benchmarks GPT-3.5 Turbo, GPT-4o, and Llama 3 70B on undergraduate circuit-analysis homework and shows reliability issues with GPT-4o when prompted simply. It proposes a three-pronged enhancement framework—multi-step prompting, context data augmentation, and targeted hints—to improve grading accuracy and reduce hallucinations. The enhanced GPT-4o achieves a substantial accuracy increase (from 74.71% to 97.70%) on entry-level topics, demonstrating the viability of LLMs as scalable teaching-assistant tools in engineering education. The study outlines a roadmap for integrating problem-context and problem-diagram information and extending the approach to broader engineering curricula.

Abstract

This research full paper presents an enhancement pipeline for large language models (LLMs) in assessing homework for an undergraduate circuit analysis course, aiming to improve LLMs' capacity to provide personalized support to electrical engineering students. Existing evaluations have demonstrated that GPT-4o possesses promising capabilities in assessing student homework in this domain. Building on these findings, we enhance GPT-4o's performance through multi-step prompting, contextual data augmentation, and the incorporation of targeted hints. These strategies effectively address common errors observed in GPT-4o's responses when using simple prompts, leading to a substantial improvement in assessment accuracy. Specifically, the correct response rate for GPT-4o increases from 74.71% to 97.70% after applying the enhanced prompting and augmented data on entry-level circuit analysis topics. This work lays a foundation for the effective integration of LLMs into circuit analysis instruction and, more broadly, into engineering education.

Enhancing Large Language Models for Automated Homework Assessment in Undergraduate Circuit Analysis

TL;DR

This work benchmarks GPT-3.5 Turbo, GPT-4o, and Llama 3 70B on undergraduate circuit-analysis homework and shows reliability issues with GPT-4o when prompted simply. It proposes a three-pronged enhancement framework—multi-step prompting, context data augmentation, and targeted hints—to improve grading accuracy and reduce hallucinations. The enhanced GPT-4o achieves a substantial accuracy increase (from 74.71% to 97.70%) on entry-level topics, demonstrating the viability of LLMs as scalable teaching-assistant tools in engineering education. The study outlines a roadmap for integrating problem-context and problem-diagram information and extending the approach to broader engineering curricula.

Abstract

This research full paper presents an enhancement pipeline for large language models (LLMs) in assessing homework for an undergraduate circuit analysis course, aiming to improve LLMs' capacity to provide personalized support to electrical engineering students. Existing evaluations have demonstrated that GPT-4o possesses promising capabilities in assessing student homework in this domain. Building on these findings, we enhance GPT-4o's performance through multi-step prompting, contextual data augmentation, and the incorporation of targeted hints. These strategies effectively address common errors observed in GPT-4o's responses when using simple prompts, leading to a substantial improvement in assessment accuracy. Specifically, the correct response rate for GPT-4o increases from 74.71% to 97.70% after applying the enhanced prompting and augmented data on entry-level circuit analysis topics. This work lays a foundation for the effective integration of LLMs into circuit analysis instruction and, more broadly, into engineering education.

Paper Structure

This paper contains 12 sections, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Preliminaries: A Benchmarking Study of Large Language Models for Undergraduate Circuit Analysis Homework Assessment
  3. Dataset Collections
  4. LLM Evaluation Metrics
  5. LLM Evaluation Results
  6. Enhancing GPT-4o for Homework Assessment
  7. Multi-Step Prompting for Different Aspects
  8. Context Data Augmentation
  9. Incorporation of Targeted Hints
  10. GPT-4o Enhancement Results
  11. Conclusions
  12. Acknowledgments

Figures (2)

  • Figure 1: Framework for evaluating and enhancing GPT-4o in undergraduate circuit analysis homework assessment. (Note: The evaluation framework shown in the top portion of this figure is adapted from Fig. 1 in chen2025benchmarking.)
  • Figure 2: Enhancement of GPT-4o in undergraduate circuit analysis homework assessment. (Note: "xxx" represents the contents of the corresponding items.)