PhysMaster: Mastering Physical Representation for Video Generation via Reinforcement Learning

TL;DR

PhysMaster tackles the gap in physics-realistic video generation by learning a physical representation from the initial image with a dedicated PhysEncoder. The representation is optimized through reinforcement learning from human feedback (Direct Preference Optimization) in a three-stage training pipeline, guiding a diffusion-based image-to-video model to produce physically plausible dynamics. Evaluations on a proxy free-fall task and broad open-world scenarios show improved physics-awareness, better generalization, and competitive efficiency compared with prior methods. The work argues for a generic, plug-in solution to physics-aware video generation with potential broad applicability.

Abstract

Video generation models nowadays are capable of generating visually realistic videos, but often fail to adhere to physical laws, limiting their ability to generate physically plausible videos and serve as ''world models''. To address this issue, we propose PhysMaster, which captures physical knowledge as a representation for guiding video generation models to enhance their physics-awareness. Specifically, PhysMaster is based on the image-to-video task where the model is expected to predict physically plausible dynamics from the input image. Since the input image provides physical priors like relative positions and potential interactions of objects in the scenario, we devise PhysEncoder to encode physical information from it as an extra condition to inject physical knowledge into the video generation process. The lack of proper supervision on the model's physical performance beyond mere appearance motivates PhysEncoder to apply reinforcement learning with human feedback to physical representation learning, which leverages feedback from generation models to optimize physical representations with Direct Preference Optimization (DPO) in an end-to-end manner. PhysMaster provides a feasible solution for improving physics-awareness of PhysEncoder and thus of video generation, proving its ability on a simple proxy task and generalizability to wide-ranging physical scenarios. This implies that our PhysMaster, which unifies solutions for various physical processes via representation learning in the reinforcement learning paradigm, can act as a generic and plug-in solution for physics-aware video generation and broader applications.

Figures (7)

• Figure 1: Overall architecture of PhysMaster. Given an input image, PhysEncoder encodes its physical feature and concatenates with visual features, then the DiT model predicts subsequent frames conditioned on physical, visual, and text embeddings. We optimize PhysEncoder’s physical representation via feedback from generated video pairs of the model by maximizing reward derived from "positive" and "negative" video outputs in a DPO paradigm.
• Figure 2: Qualitative comparison with video generation models specialized for rigid-body motion proves the advantage of our model in shape consistency and trajectory accuracy on "free-fall".
• Figure 2: Quantitative comparison with video generation models specialized for rigid-body motion verifies superiority of our model on proxy task.
• Figure 3: Qualitative ablation for models in each training stage on proxy task. The model exhibits a preliminary capability for predicting object motion trends after SFT. Two-stage DPO further improves model performance in preserving objects' rigidity and complying with physical laws (e.g., gravitational acceleration and collision). $v_\theta$ is DiT model and $E_p$ is PhysEncoder.
• Figure 4: Visualization of the first three PCA components of physical representation.$E_p$ in Stage III reveals similarities in objects under the same external forces (red: on the ground; green: in the air) over $E_p$ in Stage I.