PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions

TL;DR

PhysicsArena introduces the first holistic multimodal physics reasoning benchmark, modeling three interrelated reasoning stages—Variable Identification, Process Formulation, and Solution Derivation—to better evaluate MLLMs on physics problems. The authors assemble 5,103 high-quality multimodal problems using a four-stage data pipeline and validate annotations with GPT-4o, revealing that current models still struggle, especially with process formulation and stepwise derivation. Across a broad set of state-of-the-art open- and closed-source models, the study shows modest gains with scale and highlights persistent gaps between current capabilities and AGI-level physics reasoning. The benchmark provides a rigorous, scalable platform to drive advances in vision-language grounding and structured, domain-specific reasoning in physics.

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in diverse reasoning tasks, yet their application to complex physics reasoning remains underexplored. Physics reasoning presents unique challenges, requiring grounding in physical conditions and the interpretation of multimodal information. Current physics benchmarks are limited, often focusing on text-only inputs or solely on problem-solving, thereby overlooking the critical intermediate steps of variable identification and process formulation. To address these limitations, we introduce PhysicsArena, the first multimodal physics reasoning benchmark designed to holistically evaluate MLLMs across three critical dimensions: variable identification, physical process formulation, and solution derivation. PhysicsArena aims to provide a comprehensive platform for assessing and advancing the multimodal physics reasoning abilities of MLLMs.

Figures (22)

• Figure 1: Comparison between previous physics reasoning settings and our proposed PhysicsArena.
• Figure 2: The illustration of a representative example from our proposed PhysicsArena dataset.
• Figure 3: Roadmap of PhysicsArena dataset preparation, enhancement, and evaluation.
• Figure 4: Performance comparison for Variable Identification (a) and Process Formulation (b).
• Figure 5: Illustration of a representative bad case of variable identification (more cases in Appendix \ref{['app:case_study']}).