DiffWind: Physics-Informed Differentiable Modeling of Wind-Driven Object Dynamics

TL;DR

DiffWind is presented, a physics-informed differentiable framework that unifies wind-object interaction modeling, video-based reconstruction, and forward simulation and significantly outperforms prior dynamic scene modeling approaches in both reconstruction accuracy and simulation fidelity, opening a new avenue for video-based wind-object interaction modeling.

Abstract

Modeling wind-driven object dynamics from video observations is highly challenging due to the invisibility and spatio-temporal variability of wind, as well as the complex deformations of objects. We present DiffWind, a physics-informed differentiable framework that unifies wind-object interaction modeling, video-based reconstruction, and forward simulation. Specifically, we represent wind as a grid-based physical field and objects as particle systems derived from 3D Gaussian Splatting, with their interaction modeled by the Material Point Method (MPM). To recover wind-driven object dynamics, we introduce a reconstruction framework that jointly optimizes the spatio-temporal wind force field and object motion through differentiable rendering and simulation. To ensure physical validity, we incorporate the Lattice Boltzmann Method (LBM) as a physics-informed constraint, enforcing compliance with fluid dynamics laws. Beyond reconstruction, our method naturally supports forward simulation under novel wind conditions and enables new applications such as wind retargeting. We further introduce WD-Objects, a dataset of synthetic and real-world wind-driven scenes. Extensive experiments demonstrate that our method significantly outperforms prior dynamic scene modeling approaches in both reconstruction accuracy and simulation fidelity, opening a new avenue for video-based wind-object interaction modeling.

Figures (17)

• Figure 1: DiffWind is a physics-informed differentiable wind-object interaction framework that models wind and object dynamics separately. This design enables the reconstruction of wind and object motion from sparse-view videos, physically consistent simulation under new wind conditions, and retargeting to novel objects.
• Figure 2: Overview of DiffWind. We propose a novel wind-object interaction modeling approach, where the wind is represented as a grid field and the object is modeled as a set of particles. Based on this modeling approach, we introduce a reconstruction framework for wind–object interaction by optimizing the wind force field. In addition, we employ the Lattice Boltzmann Method (LBM) to generate wind force field direction guidance to enforce compliance with fluid dynamics laws.
• Figure 3: Our reconstruction dataset from one camera view, with PSNR values of our dynamic reconstruction shown above. Please use Adobe Reader/KDE Okular to see animations.
• Figure 4: Qualitative results of novel view synthesis for a selected synthetic wind-object interaction scene. We compare our method with Deformable-GS Deformable-Gaussian, Efficient-GS katsumata2024compact, 4D-GS Wu_2024_CVPR and GaussianPrediction zhao2024gaussianprediction. Please use Adobe Reader/KDE Okular to see animations.
• Figure 5: Wind synthesis results at the same viewpoint across different timesteps. DynamiCrafter xing2023dynamicrafter fails to maintain temporal coherence, CogVideoX yang2024cogvideox produces unrealistic jittering. In contrast, DiffWind generates realistic time-evolving wind-object interactions. Please use Adobe Reader/KDE Okular to see animations.