Exploring the potential of ChatGPT for feedback and evaluation in experimental physics

TL;DR

This study investigates the use of ChatGPT to assist in evaluating physics laboratory reports, focusing on two modalities: an API-based automated evaluator and a customized Physics Instructor ChatGPT. It analyzes performance across two criteria—Formal/Structural Integrity and Technical Precision/Conceptual Depth—using about 50 reports from Simple Pendulum and Error and Uncertainties experiments. The API-based approach reliably checks structure and writing but struggles with non-text data, while the customized model provides contextual feedback with more nuanced physics insight yet variable reproducibility and interpretive limitations. The work advocates a supervised, hybrid AI-assisted workflow to reduce instructor workload while preserving the integrity of physical reasoning in experimental physics.

Abstract

This study explores how generative artificial intelligence, specifically ChatGPT, can assist in the evaluation of laboratory reports in Experimental Physics. Two interaction modalities were implemented: an automated API-based evaluation and a customized ChatGPT configuration designed to emulate instructor feedback. The analysis focused on two complementary dimensions-formal and structural integrity, and technical accuracy and conceptual depth. Findings indicate that ChatGPT provides consistent feedback on organization, clarity, and adherence to scientific conventions, while its evaluation of technical reasoning and interpretation of experimental data remains less reliable. Each modality exhibited distinctive limitations, particularly in processing graphical and mathematical information. The study contributes to understanding how the use of AI in evaluating laboratory reports can inform feedback practices in experimental physics, highlighting the importance of teacher supervision to ensure the validity of physical reasoning and the accurate interpretation of experimental results.