Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Practice Less, Explain More: LLM-Supported Self-Explanation Improves Explanation Quality on Transfer Problems in Calculus

Eason Chen, Xinyi Tang, Yvonne Zhao, Meiyi Chen, Meryam Elmir, Elizabeth McLaughlin, Mingyu Yuan, Yumo Wang, Shyam Agarwal, Jared Cochrane, Jionghao Lin, Tongshuang Wu, Ken Koedinger

Abstract

We conducted a between-subjects experiment (N=92) comparing three conditions in a calculus learning environment: no self-explanation (control), menu-based self-explanation, and open-ended self-explanation with LLM-generated feedback. All conditions showed positive learning gains within a fixed 60-minute practice session, with no significant between-condition differences in post-test performance. On transfer questions, the open-ended condition produced significantly higher-quality explanations than control on "Not Enough Information" (NEI) problems ($β$=+11.9 percentage points, $p$=.030), though the corresponding NEI multiple-choice accuracy advantage was not significant ($p$=.183). Moreover, across all post-test open-ended explanations, the open-ended condition showed a marginally significant advantage ($β$=+7.3%, $p$=.057). These findings suggest that LLM-supported open-ended self-explanation can improve explanation quality on NEI transfer problems, with weaker evidence across broader transfer explanation measures. Notably, these effects emerged even though learners in the open-ended condition completed substantially fewer practice problems within the same practice time.

Practice Less, Explain More: LLM-Supported Self-Explanation Improves Explanation Quality on Transfer Problems in Calculus

Abstract

We conducted a between-subjects experiment (N=92) comparing three conditions in a calculus learning environment: no self-explanation (control), menu-based self-explanation, and open-ended self-explanation with LLM-generated feedback. All conditions showed positive learning gains within a fixed 60-minute practice session, with no significant between-condition differences in post-test performance. On transfer questions, the open-ended condition produced significantly higher-quality explanations than control on "Not Enough Information" (NEI) problems (=+11.9 percentage points, =.030), though the corresponding NEI multiple-choice accuracy advantage was not significant (=.183). Moreover, across all post-test open-ended explanations, the open-ended condition showed a marginally significant advantage (=+7.3%, =.057). These findings suggest that LLM-supported open-ended self-explanation can improve explanation quality on NEI transfer problems, with weaker evidence across broader transfer explanation measures. Notably, these effects emerged even though learners in the open-ended condition completed substantially fewer practice problems within the same practice time.

Paper Structure

This paper contains 5 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Results
  4. Discussion
  5. Conclusion

Figures (2)

  • Figure 1: Practice interface (control condition). Top: multiple-choice with correctness feedback and progressive hints. Bottom: short-answer with a piecewise function graph.
  • Figure 2: Self-explanation conditions. Top: menu-based, selecting from three options (one correct, two misconceptions). Bottom: open-ended with LLM feedback showing iterative revision (red = needs revision, yellow = almost there) and a reference explanation after two unsuccessful attempts.