Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

aiPlato: A Novel AI Tutoring and Step-wise Feedback System for Physics Homework

Atharva Dange, Ramon E. Lopez, Louis Deslauriers, Nimish Shah

TL;DR

aiPlato investigates AI-mediated, stepwise feedback for open-ended physics homework in a large mechanics course. The authors triangulate engagement with final-exam performance and end-of-term surveys to assess learning impact and usability. Findings show higher final-exam scores associated with greater engagement, with an estimated effect size around $d \approx 0.81$, though causality remains unestablished due to self-selection. The work demonstrates the feasibility of integrating AITS-style feedback into authentic physics homework and outlines design and methodological directions for future controlled studies.

Abstract

This exploratory study examines the classroom deployment of aiPlato, an AI-enabled homework platform, in a large introductory physics course at the University of Texas at Arlington. Designed to support open-ended problem solving, aiPlato provides step-wise feedback and iterative guidance through tools such as "Evaluate My Work" and "AI Tutor Chat", while preserving opportunities for productive struggle. Over four optional extra-credit assignments, the platform captured detailed student interaction data, which were analyzed alongside course performance and end-of-semester survey responses. We examine how students engaged with different feedback tools, whether engagement patterns were associated with performance on the cumulative final exam, and how students perceived the platform's usability and learning value. Students who engaged more frequently with aiPlato tended to achieve higher final exam scores, with a mean difference corresponding to a standardized effect size of approximately 0.81 between high and low engagement groups after controlling for prior academic performance. Usage patterns and survey responses indicate that students primarily relied on iterative, formative feedback rather than solution-revealing assistance. As a quasi-experimental pilot study, these findings do not establish causality and may reflect self-selection effects. Nonetheless, the results demonstrate the feasibility of integrating AI-mediated, step-wise feedback into authentic physics homework and motivate future controlled studies of AI-assisted tutoring systems.

aiPlato: A Novel AI Tutoring and Step-wise Feedback System for Physics Homework

TL;DR

aiPlato investigates AI-mediated, stepwise feedback for open-ended physics homework in a large mechanics course. The authors triangulate engagement with final-exam performance and end-of-term surveys to assess learning impact and usability. Findings show higher final-exam scores associated with greater engagement, with an estimated effect size around , though causality remains unestablished due to self-selection. The work demonstrates the feasibility of integrating AITS-style feedback into authentic physics homework and outlines design and methodological directions for future controlled studies.

Abstract

This exploratory study examines the classroom deployment of aiPlato, an AI-enabled homework platform, in a large introductory physics course at the University of Texas at Arlington. Designed to support open-ended problem solving, aiPlato provides step-wise feedback and iterative guidance through tools such as "Evaluate My Work" and "AI Tutor Chat", while preserving opportunities for productive struggle. Over four optional extra-credit assignments, the platform captured detailed student interaction data, which were analyzed alongside course performance and end-of-semester survey responses. We examine how students engaged with different feedback tools, whether engagement patterns were associated with performance on the cumulative final exam, and how students perceived the platform's usability and learning value. Students who engaged more frequently with aiPlato tended to achieve higher final exam scores, with a mean difference corresponding to a standardized effect size of approximately 0.81 between high and low engagement groups after controlling for prior academic performance. Usage patterns and survey responses indicate that students primarily relied on iterative, formative feedback rather than solution-revealing assistance. As a quasi-experimental pilot study, these findings do not establish causality and may reflect self-selection effects. Nonetheless, the results demonstrate the feasibility of integrating AI-mediated, step-wise feedback into authentic physics homework and motivate future controlled studies of AI-assisted tutoring systems.
Paper Structure (19 sections, 4 equations, 13 figures, 11 tables)

This paper contains 19 sections, 4 equations, 13 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Intelligent Tutoring Systems (ITS) in Physics Education
  3. LLM-Based Tutors
  4. Need for a Novel AI Platform for Physics Learning
  5. aiPlato: A Walkthrough
  6. Student Engagement and Usage Patterns
  7. Spring 2025
  8. Effect of aiPlato on Final Exam Scores
  9. Final Grade Categorical Analysis
  10. Qualitative Analysis
  11. Limitations and Future Work
  12. Engagement Metrics and Behavioral Insights
  13. Statistical Power and Model Complexity
  14. Conclusion
  15. Statistical Assumptions from \ref{['subsec4.1']}
  16. ...and 4 more sections

Figures (13)

  • Figure 1: Performance comparison of GPT-3.5 and GPT-4 (with and without vision) across standardized exams. Notably, GPT-4 scored in the 43rd percentile on AP Calculus BC (column 1) and the 66th percentile on AP Physics 2 (column 10) openaiGPT4TechnicalReport2024
  • Figure 2: General layout of the aiPlato platform as seen by students during assignments.
  • Figure 3: Handwritten input with real-time feedback and partial credit per line
  • Figure 4: aiPlato AI Tutor Chat explaining conceptual reasoning behind an error
  • Figure 5: Number of students attempting each problem in the Extra Credit 4 assignment
  • ...and 8 more figures