PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models

TL;DR

The paper tackles the lack of explicit physical realism in transformer-based video generation by introducing PhysRVG, a physics-aware reinforcement learning framework that enforces rigid-body collision rules in high-dimensional spaces. It combines Flow Matching with a physics-grounded reward, and extends it into the Mimicry-Discovery Cycle (MDcycle) to enable substantial, physics-consistent fine-tuning while preserving data-driven visual priors. A new PhysRVGBench benchmark assesses four rigid-body motions using IoU and Trajectory Offset, and extensive experiments show improvements in both visual fidelity and physical plausibility over strong baselines. The work advances practical, physics-consistent video synthesis and provides a foundation for integrating fundamental mechanics into large-scale generative models, with thoughtful discussion of limitations and ethical considerations.

Abstract

Physical principles are fundamental to realistic visual simulation, but remain a significant oversight in transformer-based video generation. This gap highlights a critical limitation in rendering rigid body motion, a core tenet of classical mechanics. While computer graphics and physics-based simulators can easily model such collisions using Newton formulas, modern pretrain-finetune paradigms discard the concept of object rigidity during pixel-level global denoising. Even perfectly correct mathematical constraints are treated as suboptimal solutions (i.e., conditions) during model optimization in post-training, fundamentally limiting the physical realism of generated videos. Motivated by these considerations, we introduce, for the first time, a physics-aware reinforcement learning paradigm for video generation models that enforces physical collision rules directly in high-dimensional spaces, ensuring the physics knowledge is strictly applied rather than treated as conditions. Subsequently, we extend this paradigm to a unified framework, termed Mimicry-Discovery Cycle (MDcycle), which allows substantial fine-tuning while fully preserving the model's ability to leverage physics-grounded feedback. To validate our approach, we construct new benchmark PhysRVGBench and perform extensive qualitative and quantitative experiments to thoroughly assess its effectiveness.

Figures (15)

• Figure 1: Samples generated by PhysRVG. Our model produces videos with physically plausible rigid body dynamics. Rows1–4 display four fundamental types of motion addressed in our work, row5 validates the model’s generalization in out-of-distribution scenarios.
• Figure 2: The core idea of PhysRVG. DiT-based video generative models reconstruct noisy videos in latent space using Flow Matching loss, which only captures data distributions ( ✓) but discards essential spatio-temporal physical cues during conditional alignment and feature extraction ( ✘), thereby hindering stable learning of physical knowledge ( ✘). While reinforcement learning with subjective ratings can train on physics-rich video data using RL-based feedback ( ✓), its evaluation remains perceptually biased and fails to provide stable physical supervision ( ✘). In contrast, our PhysRVG leverages the MDcycle to fully utilize data for visual refinement ( ✓) and enforces physical knowledge injection through the Physics-Grounded Metric ( ✓), enabling stable retention and active discovery of physical principles for truly physics-aware learning and generation ( ✓).
• Figure 3: The framework of PhysRVG. Given a text prompt and context frames, the model generates future video frames. For both the groundtruth and sampled frames, we derive motion masks $M$ by prompting SAM2 ravi2024sam2 with object coordinates $p_1$ from the first frame, which are manually annotated during data preprocessing. We then compute object trajectories $P$ and perform collision detection. The trajectory offset $O$ between the sampled and groundtruth trajectories is calculated and reweighted by the collision signal $w_t$ to yield a weighted trajectory offset $O_c$, which serves as the per-sample score. All transformer blocks are trained with full parameters.
• Figure 4: (a) RL Training loss for samples of varying quality. (b) Number of samples assigned to the Mimicry and Discovery branches throughout MDcycle.
• Figure 5: Qualitative comparisons with existing methods. Each sample in the figure corresponds to the final frame of a generated video. We include the original videos for all cases in the Supplementary Materials.