Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NotebookLM as a Socratic physics tutor: Design and preliminary observations of a RAG-based tool

Eugenio Tufino

TL;DR

The paper investigates building a grounded physics tutor using a RAG-based NotebookLM system to mitigate LLM hallucinations by grounding responses in teacher-curated sources. It details a Socratic, collaborative tutoring approach guided by a Training Manual and layered prompts, implemented in NotebookLM with a three-panel interface. Through illustrative dialogues and demonstrations with teachers, it highlights potential benefits for guided inquiry as well as pedagogical challenges like student motivation and graph-interpretation reliability. The work offers a replicable model for instructor-led, AI-assisted physics education and outlines directions for future refinements and formal evaluation.

Abstract

This study explores NotebookLM, a Google Gemini - powered AI platform that integrates Retrieval-Augmented Generation (RAG) as a Socratic tutor for physics education. In this implementation, NotebookLM was configured to support students in solving conceptually oriented physics problems through a guided, questioning-based dialogue. When deployed as a collaborative tutor, the system restricts student interaction to a chat-only interface, promoting controlled and guided engagement. By grounding its responses in teacher-provided source documents, the AI tutor helps mitigate one of the major shortcomings of standard Large Language Models - hallucinations - thereby ensuring more traceable and reliable answers. This work details the methodological design of the tutor, including the iterative development of a pedagogical "Training Manual", and presents preliminary qualitative observations from demonstrations with pre-service and in-service teachers. These observations highlight both the promising potential of the tool and key pedagogical challenges, such as managing user motivation. While limitations remain, this work offers a promising and replicable model for educators seeking to implement grounded AI tutors in their own teaching contexts.

NotebookLM as a Socratic physics tutor: Design and preliminary observations of a RAG-based tool

TL;DR

The paper investigates building a grounded physics tutor using a RAG-based NotebookLM system to mitigate LLM hallucinations by grounding responses in teacher-curated sources. It details a Socratic, collaborative tutoring approach guided by a Training Manual and layered prompts, implemented in NotebookLM with a three-panel interface. Through illustrative dialogues and demonstrations with teachers, it highlights potential benefits for guided inquiry as well as pedagogical challenges like student motivation and graph-interpretation reliability. The work offers a replicable model for instructor-led, AI-assisted physics education and outlines directions for future refinements and formal evaluation.

Abstract

This study explores NotebookLM, a Google Gemini - powered AI platform that integrates Retrieval-Augmented Generation (RAG) as a Socratic tutor for physics education. In this implementation, NotebookLM was configured to support students in solving conceptually oriented physics problems through a guided, questioning-based dialogue. When deployed as a collaborative tutor, the system restricts student interaction to a chat-only interface, promoting controlled and guided engagement. By grounding its responses in teacher-provided source documents, the AI tutor helps mitigate one of the major shortcomings of standard Large Language Models - hallucinations - thereby ensuring more traceable and reliable answers. This work details the methodological design of the tutor, including the iterative development of a pedagogical "Training Manual", and presents preliminary qualitative observations from demonstrations with pre-service and in-service teachers. These observations highlight both the promising potential of the tool and key pedagogical challenges, such as managing user motivation. While limitations remain, this work offers a promising and replicable model for educators seeking to implement grounded AI tutors in their own teaching contexts.

Paper Structure

This paper contains 7 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Design and Implementation of the Collaborative AI Tutor
  3. Defining the Tutor's Behavior: The "Training Manual"
  4. Deployment and Student Interaction Environment
  5. Content Curation and Problem Selection
  6. Illustrative examples and preliminary observations
  7. Conclusions

Figures (5)

  • Figure 1: Screenshot of the NotebookLM interface showing the three panels: Sources for storing and indexing diverse teaching materials with traceable citations; chat for dialogue; a Studio for automatically generating structured learning aids such as summaries, study guides, mind maps and podcast-style audio summaries.
  • Figure 2: NotebookLM interface: (a) Sharing options configuration available to teachers with NotebookLM Plus, now including public link sharing that allows chat-only access for students without email requirements. (b) The student chat interface with a sample welcome message.
  • Figure 3: NotebookLM's graph interpretation across formats. (a) Velocity--time graph for the bouncing ball. (b) Native Google Doc: system correctly reads the graph and returns the requested numerical values. (c) PDF exported from Google Docs: the figure is not accessible; the model reports that the graph is not present in the source and cannot extract axes or numeric values.
  • Figure 4: Schematic of the DC circuit with two parallel resistors discussed in the problem.
  • Figure 5: A block remains stationary against the back wall of an accelerating cart. Problem adapted from etkina2019college.