Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Augmented Physics: Creating Interactive and Embedded Physics Simulations from Static Textbook Diagrams

Aditya Gunturu, Yi Wen, Nandi Zhang, Jarin Thundathil, Rubaiat Habib Kazi, Ryo Suzuki

TL;DR

Augmented Physics tackles the gap between static physics textbook diagrams and interactive learning by converting diagrams into embedded simulations using Segment-Anything and multimodal LLMs. The system defines four augmentation strategies, implements a web-based authoring workflow, and demonstrates kinematics, optics, and circuit simulations with a hybrid backend/frontend pipeline. Technical evaluation (200 diagrams across six textbooks) and user studies (N=12 for both usability and expert interviews) show high segmentation success, reasonable simulation rates, and strong user acceptance, suggesting the approach can personalize and enliven physics education when used as a complement to existing resources. The work highlights practical opportunities and challenges for classroom deployment, including accuracy verification, on-demand content creation, and potential extensions to broader topics and AR-enabled experiences.

Abstract

We introduce Augmented Physics, a machine learning-integrated authoring tool designed for creating embedded interactive physics simulations from static textbook diagrams. Leveraging recent advancements in computer vision, such as Segment Anything and Multi-modal LLMs, our web-based system enables users to semi-automatically extract diagrams from physics textbooks and generate interactive simulations based on the extracted content. These interactive diagrams are seamlessly integrated into scanned textbook pages, facilitating interactive and personalized learning experiences across various physics concepts, such as optics, circuits, and kinematics. Drawing from an elicitation study with seven physics instructors, we explore four key augmentation strategies: 1) augmented experiments, 2) animated diagrams, 3) bi-directional binding, and 4) parameter visualization. We evaluate our system through technical evaluation, a usability study (N=12), and expert interviews (N=12). Study findings suggest that our system can facilitate more engaging and personalized learning experiences in physics education.

Augmented Physics: Creating Interactive and Embedded Physics Simulations from Static Textbook Diagrams

TL;DR

Augmented Physics tackles the gap between static physics textbook diagrams and interactive learning by converting diagrams into embedded simulations using Segment-Anything and multimodal LLMs. The system defines four augmentation strategies, implements a web-based authoring workflow, and demonstrates kinematics, optics, and circuit simulations with a hybrid backend/frontend pipeline. Technical evaluation (200 diagrams across six textbooks) and user studies (N=12 for both usability and expert interviews) show high segmentation success, reasonable simulation rates, and strong user acceptance, suggesting the approach can personalize and enliven physics education when used as a complement to existing resources. The work highlights practical opportunities and challenges for classroom deployment, including accuracy verification, on-demand content creation, and potential extensions to broader topics and AR-enabled experiences.

Abstract

We introduce Augmented Physics, a machine learning-integrated authoring tool designed for creating embedded interactive physics simulations from static textbook diagrams. Leveraging recent advancements in computer vision, such as Segment Anything and Multi-modal LLMs, our web-based system enables users to semi-automatically extract diagrams from physics textbooks and generate interactive simulations based on the extracted content. These interactive diagrams are seamlessly integrated into scanned textbook pages, facilitating interactive and personalized learning experiences across various physics concepts, such as optics, circuits, and kinematics. Drawing from an elicitation study with seven physics instructors, we explore four key augmentation strategies: 1) augmented experiments, 2) animated diagrams, 3) bi-directional binding, and 4) parameter visualization. We evaluate our system through technical evaluation, a usability study (N=12), and expert interviews (N=12). Study findings suggest that our system can facilitate more engaging and personalized learning experiences in physics education.
Paper Structure (40 sections, 12 figures, 1 table)

This paper contains 40 sections, 12 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Physics Simulation for Learning
  4. Augmenting Existing Documents
  5. Tools for Authoring Interactive Diagrams
  6. Formative Study
  7. Method
  8. Participants
  9. Protocol
  10. Challenges of Current Practices
  11. Static Visualizations Cannot Represent Time-Dependent Physics Concepts
  12. Videos Enhance Understanding but Lack Experimentation Opportunities
  13. Simulation Tools Lack Sufficient Instructional Scaffolding
  14. External Content Might Misalign and Distract from Core Learning
  15. Elicited Augmentation Strategies
  16. ...and 25 more sections

Figures (12)

  • Figure 1: Augmented Experiments: Enabling users to directly manipulate textbook diagrams, enabling them to change parameters such as the position of an object in an optics diagram or the resistance in a circuit diagram, and observe real-time changes.
  • Figure 2: Animated Diagrams: Converting static figures into looped dynamic animations, showing changes over time.
  • Figure 3: Bi-Directional Binding: Connecting text to the diagrams and making them manipulable.
  • Figure 4: Parameter Visualization: Generating on-demand visualizations of various parameters in the simulated diagram.
  • Figure 5: Interactive simulations for an optics diagram. 1) The user segments objects, lenses, and focal points. 2) The system generates an overlaid simulation. 3) The user interacts with the object and focal point to observe changes.
  • ...and 7 more figures