Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Creating a customisable Socratic AI physics tutor

Eugenio Tufino, Bor Gregorcic

TL;DR

Role engineering via Gemini Gems enables tailoring LLMs into specialized physics tutors by scripting a pedagogical persona and grounding with Knowledge documents. The paper demonstrates two illustrative use cases: multimodal feedback on force diagrams using course-specific notation and electromagnetism reasoning guided by pre-trained knowledge. The results show that a role-engineered Gem can sustain a Socratic dialogue and scaffold step-by-step problem solving, in contrast to unmodified models that output direct solutions, with a key formula such as $\mathcal{E} = B \cdot L \cdot v$ featured in the EM case. While promising as an accessible path for educators to create tailored learning tools, the work remains qualitative and acknowledges potential inaccuracies, underscoring the need for systematic evaluation and expansion to other physics roles.

Abstract

This paper explores role engineering as an effective paradigm for customizing Large Language Models (LLMs) into specialized AI tutors for physics education. We demonstrate this methodology by designing a Socratic physics problem-solving tutor using Google's Gemini Gems feature, defining its pedagogical behavior through a detailed 'script' that specifies its role and persona. We present two illustrative use cases: the first demonstrates the Gem's multimodal ability to analyze a student's hand-drawn force diagram and apply notational rules from a 'Knowledge' file; the second showcases its capacity to guide conceptual reasoning in electromagnetism using its pre-trained knowledge without using specific documents provided by the instructor. Our findings show that the 'role-engineered' Gem successfully facilitates a Socratic dialogue, in stark contrast to a standard Gemini model, which tends to immediately provide direct solutions. We conclude that role engineering is a pivotal and accessible method for educators to transform a general-purpose 'solution provider' into a reliable pedagogical tutor capable of engaging students in an active reflection process. This approach offers a powerful tool for both instructors and students, while also highlighting the importance of addressing the technology's inherent limitations, such as the potential for occasional inaccuracies.

Creating a customisable Socratic AI physics tutor

TL;DR

Role engineering via Gemini Gems enables tailoring LLMs into specialized physics tutors by scripting a pedagogical persona and grounding with Knowledge documents. The paper demonstrates two illustrative use cases: multimodal feedback on force diagrams using course-specific notation and electromagnetism reasoning guided by pre-trained knowledge. The results show that a role-engineered Gem can sustain a Socratic dialogue and scaffold step-by-step problem solving, in contrast to unmodified models that output direct solutions, with a key formula such as featured in the EM case. While promising as an accessible path for educators to create tailored learning tools, the work remains qualitative and acknowledges potential inaccuracies, underscoring the need for systematic evaluation and expansion to other physics roles.

Abstract

This paper explores role engineering as an effective paradigm for customizing Large Language Models (LLMs) into specialized AI tutors for physics education. We demonstrate this methodology by designing a Socratic physics problem-solving tutor using Google's Gemini Gems feature, defining its pedagogical behavior through a detailed 'script' that specifies its role and persona. We present two illustrative use cases: the first demonstrates the Gem's multimodal ability to analyze a student's hand-drawn force diagram and apply notational rules from a 'Knowledge' file; the second showcases its capacity to guide conceptual reasoning in electromagnetism using its pre-trained knowledge without using specific documents provided by the instructor. Our findings show that the 'role-engineered' Gem successfully facilitates a Socratic dialogue, in stark contrast to a standard Gemini model, which tends to immediately provide direct solutions. We conclude that role engineering is a pivotal and accessible method for educators to transform a general-purpose 'solution provider' into a reliable pedagogical tutor capable of engaging students in an active reflection process. This approach offers a powerful tool for both instructors and students, while also highlighting the importance of addressing the technology's inherent limitations, such as the potential for occasional inaccuracies.

Paper Structure

This paper contains 6 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Pedagogical Design
  3. Illustrative Use Cases
  4. Illustrative Case 1: Multimodal Feedback on Force Diagrams
  5. Illustrative Case: Spatial and quantitative Reasoning in Electromagnetic Induction
  6. Conclusions

Figures (3)

  • Figure 1: The configuration interface for a custom Gemini Gem. The key components are shown: the "Instructions" panel for defining the AI's role and persona, the "Knowledge" section for uploading supplementary grounding documents, and the "Preview" window for testing and iteratively refining the Gem's responses.
  • Figure 2: Student's hand-drawn force diagrams for the stunt performer problem. (a) The initial attempt, using common but simplified notation (e.g., 'W' for weight). (b) The revised diagram after interaction with the AI tutor, which correctly applies the required two-subscript notation convention ($F_{A\text{ on }S}$).
  • Figure 3: The setup for the quantitative problem on electromagnetic induction, discussed in Case Study 2. A rectangular conducting loop enters a region of uniform magnetic field at a constant velocity.