Let it Snow! Animating 3D Gaussian Scenes with Dynamic Weather Effects via Physics-Guided Score Distillation

TL;DR

This work tackles dynamic editing of static 3D Gaussian Splatting scenes by introducing Physics-Guided Score Distillation, which uses Material Point Method-based physics as a motion prior to guide Video Score Distillation Sampling for joint motion and appearance optimization. A recurrent neural dynamics model, physics regularization, and SDS-adaptive weighting enable scene-wide, multi-particle weather effects such as snowfall, rainfall, fog, and sandstorms with continuous emission. The approach yields photorealistic, temporally coherent dynamics that outperform static editing baselines and recent 4D editing methods, demonstrating the value of integrating physics priors with diffusion-based appearance refinement for dynamic 3D content. This framework bridges physics simulation and diffusion priors to enable realistic, scene-wide dynamic weather editing in 3D Gaussian scenes.

Abstract

3D Gaussian Splatting has recently enabled fast and photorealistic reconstruction of static 3D scenes. However, dynamic editing of such scenes remains a significant challenge. We introduce a novel framework, Physics-Guided Score Distillation, to address a fundamental conflict: physics simulation provides a strong motion prior that is insufficient for photorealism , while video-based Score Distillation Sampling (SDS) alone cannot generate coherent motion for complex, multi-particle scenarios. We resolve this through a unified optimization framework where physics simulation guides Score Distillation to jointly refine the motion prior for photorealism while simultaneously optimizing appearance. Specifically, we learn a neural dynamics model that predicts particle motion and appearance, optimized end-to-end via a combined loss integrating Video-SDS for photorealism with our physics-guidance prior. This allows for photorealistic refinements while ensuring the dynamics remain plausible. Our framework enables scene-wide dynamic weather effects, including snowfall, rainfall, fog, and sandstorms, with physically plausible motion. Experiments demonstrate our physics-guided approach significantly outperforms baselines, with ablations confirming this joint refinement is essential for generating coherent, high-fidelity dynamics.

Figures (11)

• Figure 1: Overview. Left: Physics-Based Motion Prior. Given multi-view images of a static scene, we reconstruct the scene using 3DGS, map the static Gaussians to a particle representation, introduce dynamic particles, and simulate their motion using the Material Point Method (MPM). We then map these particles back to the Gaussian world and refine scene interactions with our mesh-based collision handling technique. The simulation provides reference motion trajectories that serve as guidance prior. Right: Physics-Guided Score Distillation. Our Neural Dynamics Model is conditioned on reference trajectories from physics simulation and optimized through Video-SDS with physics regularization losses. This joint optimization refines motion for photorealism while synthesizing appearance, maintaining physical plausibility through physics guidance. This produces photorealistic dynamic weather effects with continuous particle emission that integrate seamlessly with the static scene.
• Figure 2: Dynamic weather effects created with our physics-guided SDS framework across different scenes (input frames on left). Each column represents an increasing timestep, demonstrating temporal evolution with continuous particle emission and realistic scene interactions. Our approach enables scene-wide modifications where emitted particles interact naturally with static geometry and evolve coherently over time. The snowfall effect is presented in Fig. \ref{['fig:teaser']}. As the rainfall effect is challenging to visualize, we provide zoomed-in regions to better illustrate the effect. Complete temporal visualizations for all effects are available in our project webpage.
• Figure 3: Text Prompt Results. Our physics-guided framework supports creative text prompts beyond standard weather. The method successfully synthesizes the target appearance while maintaining the physically plausible motion learned for the corresponding effect (e.g., sandstorm, snowfall).
• Figure 4: Qualitative comparison between our dynamic approach and static editing. We show two different effects across separate rows, comparing the original scene, ClimateNeRF li2023climatenerf, GaussCtrl wu2024gaussctrl, and our method at both early (Small t) and later (Large t) timesteps.
• Figure 5: Qualitative Ablation Study. We compare our full method against four variants. (w/o CH) Gaussians float above surfaces without collision handling; (w/o Motion) data-driven optimization alone yields incoherent and physically implausible motion; (w/o App) physics-only achieves plausible motion but lacks photorealistic appearance; (w/o PG) fixed physics motion without joint optimization prevents effective Video-SDS optimization, hindering appearance quality. Our full method achieves both physically plausible motion and photorealistic appearance through physics-guided score distillation. Highlighted regions contrast the successful surface accumulation in our method with the floating artifacts in w/o CH.