PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models

TL;DR

PhysicsMind tackles the gap in physical understanding within contemporary multimodal foundations by introducing a unified benchmark with paired real and simulated data across Center of Mass, Lever Equilibrium, and Newton's First Law. It jointly evaluates perception and generation through VQA and Video Generation tasks, using physics-aware metrics that test law-consistent reasoning and trajectories. Across 24 vision-language models and multiple video generators, the study reveals systematic gaps: models rely on appearance priors, struggle with torque and inertia reasoning, and exhibit sim-to-real transfer challenges. The benchmark provides a targeted framework and accompanying data to drive physics-aware improvements in multimodal systems, with final data release contingent on acceptance.

Abstract

Modern foundational Multimodal Large Language Models (MLLMs) and video world models have advanced significantly in mathematical, common-sense, and visual reasoning, but their grasp of the underlying physics remains underexplored. Existing benchmarks attempting to measure this matter rely on synthetic, Visual Question Answer templates or focus on perceptual video quality that is tangential to measuring how well the video abides by physical laws. To address this fragmentation, we introduce PhysicsMind, a unified benchmark with both real and simulation environments that evaluates law-consistent reasoning and generation over three canonical principles: Center of Mass, Lever Equilibrium, and Newton's First Law. PhysicsMind comprises two main tasks: i) VQA tasks, testing whether models can reason and determine physical quantities and values from images or short videos, and ii) Video Generation(VG) tasks, evaluating if predicted motion trajectories obey the same center-of-mass, torque, and inertial constraints as the ground truth. A broad range of recent models and video generation models is evaluated on PhysicsMind and found to rely on appearance heuristics while often violating basic mechanics. These gaps indicate that current scaling and training are still insufficient for robust physical understanding, underscoring PhysicsMind as a focused testbed for physics-aware multimodal models. Our data will be released upon acceptance.

Figures (10)

• Figure 1: Three canonical mechanics scenarios in PhysicsMind: Center of Mass, lever equilibrium, and Newton’s first law, each realized with various real tabletop and simulated configurations.
• Figure 2: Overview of the PhysicsMind framework. It combines a foundational model with physics-guided dataset construction, expert-verified annotations, and diverse controlled scenarios to enable robust video understanding and physics-aware evaluation.
• Figure 3: Physics-Aware evaluation metrics for Video Generation Models (VGM). Inertia metrics assess motion and trajectory consistency, Center‑of‑Mass metrics measure segmentation alignment, and Lever‑Equilibrium evaluates final‑state agreement.
• Figure 4: Error analysis of visual reasoning and video generation. Left: Gemini 2.5 Pro correctly predicts lever balance, while Claude 4.5 gives an incorrect prediction. Right: Sora 2 generates physically consistent motion per Newton’s First Law, unlike LTX-Video's generation
• Figure 5: Visual overview of dataset breadth and variation across real and simulated settings.