Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PhysicsSolutionAgent: Towards Multimodal Explanations for Numerical Physics Problem Solving

Aditya Thole, Anmol Agrawal, Arnav Ramamoorthy, Dhruv Kumar

TL;DR

This work addresses the need for multimodal explanations in numerical physics by introducing PhysicsSolutionAgent (PSA), a multi-stage agentic pipeline that generates long-form Manim videos with plan, code, execution, and visual refinement. It combines Retrieval-Augmented Generation and a Vision-Language feedback loop to ground code generation, debug, and polish visual outputs, evaluated via a 15-metric automated rubric and human analysis. Across conceptual and numerical tasks, PSA achieves about 3.8/5 average quality with consistent performance, while identifying recurring issues such as narration redundancy and visual layout inconsistencies. The study highlights both the potential and current limitations of automated, multimodal educational content and points to future work on verification, conciseness, and real-time capabilities.

Abstract

Explaining numerical physics problems often requires more than text-based solutions; clear visual reasoning can substantially improve conceptual understanding. While large language models (LLMs) demonstrate strong performance on many physics questions in textual form, their ability to generate long, high-quality visual explanations remains insufficiently explored. In this work, we introduce PhysicsSolutionAgent (PSA), an autonomous agent that generates physics-problem explanation videos of up to six minutes using Manim animations. To evaluate the generated videos, we design an assessment pipeline that performs automated checks across 15 quantitative parameters and incorporates feedback from a vision-language model (VLM) to iteratively improve video quality. We evaluate PSA on 32 videos spanning numerical and theoretical physics problems. Our results reveal systematic differences in video quality depending on problem difficulty and whether the task is numerical or theoretical. Using GPT-5-mini, PSA achieves a 100% video-completion rate with an average automated score of 3.8/5. However, qualitative analysis and human inspection uncover both minor and major issues, including visual layout inconsistencies and errors in how visual content is interpreted during feedback. These findings expose key limitations in reliable Manim code generation and highlight broader challenges in multimodal reasoning and evaluation for visual explanations of numerical physics problems. Our work underscores the need for improved visual understanding, verification, and evaluation frameworks in future multimodal educational systems

PhysicsSolutionAgent: Towards Multimodal Explanations for Numerical Physics Problem Solving

TL;DR

This work addresses the need for multimodal explanations in numerical physics by introducing PhysicsSolutionAgent (PSA), a multi-stage agentic pipeline that generates long-form Manim videos with plan, code, execution, and visual refinement. It combines Retrieval-Augmented Generation and a Vision-Language feedback loop to ground code generation, debug, and polish visual outputs, evaluated via a 15-metric automated rubric and human analysis. Across conceptual and numerical tasks, PSA achieves about 3.8/5 average quality with consistent performance, while identifying recurring issues such as narration redundancy and visual layout inconsistencies. The study highlights both the potential and current limitations of automated, multimodal educational content and points to future work on verification, conciseness, and real-time capabilities.

Abstract

Explaining numerical physics problems often requires more than text-based solutions; clear visual reasoning can substantially improve conceptual understanding. While large language models (LLMs) demonstrate strong performance on many physics questions in textual form, their ability to generate long, high-quality visual explanations remains insufficiently explored. In this work, we introduce PhysicsSolutionAgent (PSA), an autonomous agent that generates physics-problem explanation videos of up to six minutes using Manim animations. To evaluate the generated videos, we design an assessment pipeline that performs automated checks across 15 quantitative parameters and incorporates feedback from a vision-language model (VLM) to iteratively improve video quality. We evaluate PSA on 32 videos spanning numerical and theoretical physics problems. Our results reveal systematic differences in video quality depending on problem difficulty and whether the task is numerical or theoretical. Using GPT-5-mini, PSA achieves a 100% video-completion rate with an average automated score of 3.8/5. However, qualitative analysis and human inspection uncover both minor and major issues, including visual layout inconsistencies and errors in how visual content is interpreted during feedback. These findings expose key limitations in reliable Manim code generation and highlight broader challenges in multimodal reasoning and evaluation for visual explanations of numerical physics problems. Our work underscores the need for improved visual understanding, verification, and evaluation frameworks in future multimodal educational systems
Paper Structure (22 sections, 4 figures, 1 table)

This paper contains 22 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related works
  3. Automated Generation of Multimodal Explanations
  4. LLMs for code-driven animations
  5. Agentic Frameworks for Scientific Tasks
  6. Methodology
  7. Stage 1: Solution Generation
  8. Stage 2: Scene Planning
  9. Stage 3: RAG-Enhanced Code Generation
  10. Stage 4: Execution with Error-Correction Loop
  11. Stage 5: Screenshot-Driven Visual Refinement
  12. Experimental Setup
  13. Dataset and Task Categorization
  14. Automated Evaluation Framework
  15. Data Logging and Qualitative Analysis
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Comparison between a traditional text-based physics solution and the multimodal solution generated by PhysicsSolutionAgent, showing how visual explanations improve clarity and intuition.
  • Figure 2: The end-to-end video generation pipeline. Specialized agents collaborate to solve the physics problem, plan the visual narrative, and generate code.
  • Figure 3: Examples of poor quality scenes.
  • Figure 4: Examples of high quality scenes.