EmbodieDreamer: Advancing Real2Sim2Real Transfer for Policy Training via Embodied World Modeling

TL;DR

Real2Sim2Real transfer remains a major bottleneck for policy learning in embodied AI due to dynamics and appearance gaps. EmbodieDreamer addresses this by coupling PhysAligner, a differentiable physics parameter optimization module, with VisAligner, a conditional video diffusion model that translates low-fidelity simulations into photorealistic videos, enabling high-fidelity training environments. The approach yields a 3.74% reduction in physical-parameter estimation error and an 89.91% speedup over simulated-annealing baselines, plus a 29.17% improvement in average real-world task success after reinforcement learning in photorealistic environments. These results demonstrate practical gains for both RL and imitation learning, offering a scalable path toward deploying simulation-trained policies in real-world robotics.

Abstract

The rapid advancement of Embodied AI has led to an increasing demand for large-scale, high-quality real-world data. However, collecting such embodied data remains costly and inefficient. As a result, simulation environments have become a crucial surrogate for training robot policies. Yet, the significant Real2Sim2Real gap remains a critical bottleneck, particularly in terms of physical dynamics and visual appearance. To address this challenge, we propose EmbodieDreamer, a novel framework that reduces the Real2Sim2Real gap from both the physics and appearance perspectives. Specifically, we propose PhysAligner, a differentiable physics module designed to reduce the Real2Sim physical gap. It jointly optimizes robot-specific parameters such as control gains and friction coefficients to better align simulated dynamics with real-world observations. In addition, we introduce VisAligner, which incorporates a conditional video diffusion model to bridge the Sim2Real appearance gap by translating low-fidelity simulated renderings into photorealistic videos conditioned on simulation states, enabling high-fidelity visual transfer. Extensive experiments validate the effectiveness of EmbodieDreamer. The proposed PhysAligner reduces physical parameter estimation error by 3.74% compared to simulated annealing methods while improving optimization speed by 89.91\%. Moreover, training robot policies in the generated photorealistic environment leads to a 29.17% improvement in the average task success rate across real-world tasks after reinforcement learning. Code, model and data will be publicly available.

Figures (6)

• Figure 2: EmbodieDreamer framework integrates PhysAligner and VisAligner to reduce the Real2Sim2Real gap in physics and appearance. PhysAligner optimizes simulator dynamics, while VisAligner translates simulated renderings into realistic observations for robot policy training.
• Figure 3: The figure illustrates the workflow of PhysAligner. First, a large amount of data is generated using a simulator. Then, a surrogate model is trained to fit the data. Finally, the physical parameters are optimized through gradient descent.
• Figure 4: The framework of VisAligner. A reference image containing the initial background and robot appearance information serves as the first frame of the conditioned video. The subsequent frames are generated by performing pixel-wise addition of the robot's motion observations from the simulated environment and the segmentation masks of the foreground objects. These frames are then encoded into latents via a VAE encoder vae, concatenated with noise along the channel dimension, and input to VisAligner for denoising. Spatial-temporal attention mechanisms are employed to capture long-range dependencies across both spatial and temporal dimensions, thereby enhancing the coherence and visual quality of the generated video. The final video is obtained by decoding the denoised latents.
• Figure 5: The visualization comparison of whether foreground object segmentation is used as a conditioning input. The four rows show: (1) Ground Truth (GT), (2) generated without segmentation condition, (3) segmentation map used as condition, and (4) generated result with segmentation condition.
• Figure 6: Visualization of two distinct trajectories generated by the policy model from a shared initial frame, after being rendered using VisAligner. Despite originating from the same starting image, the trajectories lead to different simulation states due to variations in action sequences. Thanks to the accurate robot positioning in the simulated environment, the resulting photorealistic observations can be effectively used for further policy inference.