PhysMotion: Physics-Grounded Dynamics From a Single Image

TL;DR

PhysMotion addresses single-image video generation by fusing physics-based 3D Gaussian Splatting with Material Point Method dynamics and a diffusion-based video refinement that enforces temporal coherence. The method lifts a single image to a geometry-aware 3D representation, simulates elastoplastic and viscoplastic dynamics under applied forces, and refines the result with cross-frame-attention diffusion to preserve detail. Key contributions include the first single-image workflow with 3D geometry-aware, physics-grounded dynamics and a two-stage video enhancement pipeline validated against baselines with quantitative and qualitative gains in physical plausibility and visual fidelity. This framework enables realistic, controllable dynamics from minimal input, with potential applications in animation, AR/VR, and interactive media.

Abstract

We introduce PhysMotion, a novel framework that leverages principled physics-based simulations to guide intermediate 3D representations generated from a single image and input conditions (e.g., applied force and torque), producing high-quality, physically plausible video generation. By utilizing continuum mechanics-based simulations as a prior knowledge, our approach addresses the limitations of traditional data-driven generative models and result in more consistent physically plausible motions. Our framework begins by reconstructing a feed-forward 3D Gaussian from a single image through geometry optimization. This representation is then time-stepped using a differentiable Material Point Method (MPM) with continuum mechanics-based elastoplasticity models, which provides a strong foundation for realistic dynamics, albeit at a coarse level of detail. To enhance the geometry, appearance and ensure spatiotemporal consistency, we refine the initial simulation using a text-to-image (T2I) diffusion model with cross-frame attention, resulting in a physically plausible video that retains intricate details comparable to the input image. We conduct comprehensive qualitative and quantitative evaluations to validate the efficacy of our method. Our project page is available at: https://supertan0204.github.io/physmotion_website/.

Figures (10)

• Figure 1: PhysMotion is a novel image-to-video framework that uses physical dynamics and 3D geometric representations to generate realistic videos, capturing rigid body motion, elastic deformation, viscoplastic flow, and fracture. Each image pair shows the input image on the left and the generated dynamics on the right.
• Figure 2: Overview. Given a single image input $\mathbf{I}^0$, we introduce a novel pipeline to generate high-fidelity, physics-grounded video with 3D understanding. Our pipeline consists of two main stages: first, we perform a single-view 3DGS reconstruction of the segmented object from the input image, then synthesize a physics-grounded coarse object dynamics $\{\mathcal{I}_j\}_{j=1}^N$ (\ref{['sec:geometry-aware reconstruction']}); then, we apply diffusion-based video enhancement to produce the final enhanced video $\{\mathcal{I}^*_j\}_{j=1}^N$ with backgrounds (\ref{['sec:generative-video-enhancement']}), enabling users to create visually compelling, physics-driven video from a single image with an applied conditional force or torque.
• Figure 3: Applying blending in latent space causes temporal inconsistency at cake's edge.
• Figure 4: Showcases. We demonstrate exceptional versatility of our approach across a wide variety of examples.
• Figure 5: Qualitative Comparison. We use compare our results against MotionI2V shi2024motion, DragAnything (DA) wu2025draganything, CogVideoX-5B yang2024cogvideox, DynamiCrafter (DC) xing2025dynamicrafter and I2VGen-XL shi2024motion. Text prompts for CogVideoX-5B, I2VGen-XL and DynamiCrafter are generated using ChatGPT-4o, while trajectories are used for DragAnything and Motion-I2V.