Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interpretability Framework for LLMs in Undergraduate Calculus

Sagnik Dakshit, Sushmita Sinha Roy

TL;DR

This work addresses the gap that accuracy alone fails to capture the quality and pedagogy of LLM-assisted mathematics problem solving. It introduces a domain-grounded interpretability framework that jointly analyzes structured reasoning flows and input sensitivity through prompt ablation, and applies it to real Calculus I–III exams, including retrieval-augmented strategies (RAG) and contextual retrieval. Key findings show that fluent solutions often mask conceptual gaps and that prompt phrasing strongly influences reasoning, with context-grounded retrieval offering gains in interpretability and robustness under certain conditions. The framework provides actionable insights for curriculum alignment, formative feedback, and safe deployment of AI in STEM education, with potential extensions to other disciplines. Overall, the work contributes a quantitative, pedagogically anchored approach to diagnosing and improving AI-assisted learning tools beyond simple correctness metrics.

Abstract

Large Language Models (LLMs) are increasingly being used in education, yet their correctness alone does not capture the quality, reliability, or pedagogical validity of their problem-solving behavior, especially in mathematics, where multistep logic, symbolic reasoning, and conceptual clarity are critical. Conventional evaluation methods largely focus on final answer accuracy and overlook the reasoning process. To address this gap, we introduce a novel interpretability framework for analyzing LLM-generated solutions using undergraduate calculus problems as a representative domain. Our approach combines reasoning flow extraction and decomposing solutions into semantically labeled operations and concepts with prompt ablation analysis to assess input salience and output stability. Using structured metrics such as reasoning complexity, phrase sensitivity, and robustness, we evaluated the model behavior on real Calculus I to III university exams. Our findings revealed that LLMs often produce syntactically fluent yet conceptually flawed solutions, with reasoning patterns sensitive to prompt phrasing and input variation. This framework enables fine-grained diagnosis of reasoning failures, supports curriculum alignment, and informs the design of interpretable AI-assisted feedback tools. This is the first study to offer a structured, quantitative, and pedagogically grounded framework for interpreting LLM reasoning in mathematics education, laying the foundation for the transparent and responsible deployment of AI in STEM learning environments.

Interpretability Framework for LLMs in Undergraduate Calculus

TL;DR

This work addresses the gap that accuracy alone fails to capture the quality and pedagogy of LLM-assisted mathematics problem solving. It introduces a domain-grounded interpretability framework that jointly analyzes structured reasoning flows and input sensitivity through prompt ablation, and applies it to real Calculus I–III exams, including retrieval-augmented strategies (RAG) and contextual retrieval. Key findings show that fluent solutions often mask conceptual gaps and that prompt phrasing strongly influences reasoning, with context-grounded retrieval offering gains in interpretability and robustness under certain conditions. The framework provides actionable insights for curriculum alignment, formative feedback, and safe deployment of AI in STEM education, with potential extensions to other disciplines. Overall, the work contributes a quantitative, pedagogically anchored approach to diagnosing and improving AI-assisted learning tools beyond simple correctness metrics.

Abstract

Large Language Models (LLMs) are increasingly being used in education, yet their correctness alone does not capture the quality, reliability, or pedagogical validity of their problem-solving behavior, especially in mathematics, where multistep logic, symbolic reasoning, and conceptual clarity are critical. Conventional evaluation methods largely focus on final answer accuracy and overlook the reasoning process. To address this gap, we introduce a novel interpretability framework for analyzing LLM-generated solutions using undergraduate calculus problems as a representative domain. Our approach combines reasoning flow extraction and decomposing solutions into semantically labeled operations and concepts with prompt ablation analysis to assess input salience and output stability. Using structured metrics such as reasoning complexity, phrase sensitivity, and robustness, we evaluated the model behavior on real Calculus I to III university exams. Our findings revealed that LLMs often produce syntactically fluent yet conceptually flawed solutions, with reasoning patterns sensitive to prompt phrasing and input variation. This framework enables fine-grained diagnosis of reasoning failures, supports curriculum alignment, and informs the design of interpretable AI-assisted feedback tools. This is the first study to offer a structured, quantitative, and pedagogically grounded framework for interpreting LLM reasoning in mathematics education, laying the foundation for the transparent and responsible deployment of AI in STEM learning environments.
Paper Structure (36 sections, 1 figure, 3 tables)

This paper contains 36 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Motivation: Need for Reasoning Centered Interpretability
  3. Calculus as a Case Study
  4. Contributions
  5. Related Works
  6. Explainable AI in Education
  7. Reasoning Chains in NLP
  8. LLMs in Mathematics
  9. Interpretability Framework and Quantitative Evaluation
  10. Structured Reasoning Flow Analysis
  11. Sensitivity Ablation and Input Salience
  12. Quantitative Metrics and Aggregation
  13. Experimental Setup
  14. Real World Data
  15. Grading Rubric
  16. ...and 21 more sections

Figures (1)

  • Figure 1: Flow Reasoning Framework: A visual overview of the step wise process used to analyze LLM generated responses, including operation and concept extraction, reasoning pattern detection, and ablation based prompt sensitivity analysis.