Fluid Control with Localized Spacetime Windows

TL;DR

This work tackles the challenge of efficiently controlling large-scale fluid simulations by restricting optimization to localized spacetime windows around user edits. It introduces a floating background grid to represent control forces, decoupling control from the particle-based forward solver, and uses CMA-ES to automatically select an effective temporal window within a user-defined spatial region. A unified optimization objective combines editing goals, force regularization, and a buffer constraint, enabling multiple editing modalities such as keyframes, splash edits, and pathlines with substantial performance gains over full-domain optimization. The approach demonstrates real-time-like editing capabilities for 2D and 3D free-surface flows while preserving physical plausibility and providing a flexible framework adaptable to other differentiable solvers.

Abstract

We present a physics-based fluid control method utilizing localized spacetime windows, extending force-based spacetime control to simulation scales that were previously intractable. Building on the observation that optimal control force distributions are often localized, we show that operating only in a localized spacetime window around the edit of interest can improve performance. To determine the optimal spacetime window size, we employ the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) method to search for the optimal temporal window size within a user-defined spatial region. Instead of using a Lagrangian representation, we optimize and apply control forces on a "floating" background grid, decoupling the control dimensionality from the simulation and enabling seamless integration with particle-based methods. Moreover, since the boundary conditions of the localized areas are encoded in the objective function, no extra effort is required to ensure consistency between the local control region and the global simulation domain. We demonstrate the effectiveness and efficiency of our method with various 2D and 3D particle-based free-surface simulation examples.

Figures (10)

• Figure 1: Overview of our control pipeline: Our control framework operates in three stages: 1) a forward global simulation phase with a differentiable fluid solver, 2) a hybrid localized control phase that selectively optimizes Eulerian control forces on a background grid to achieve user-specified goals, and 3) a global re-simulation phase to blend the optimal control forces into the original simulation. It supports keyframe-based control, splash editing, and pathline control as shown in (b), all within a unified control formulation. Multiple localized control forces can be reintegrated into the global simulation to generate a modified animation as demonstrated in (d).
• Figure 2: Localized control. Given a requested local edit (the tip of the splash should move), the solution of a global grid-based spacetime optimization (b) is localized. Restricting the control problem to a local window from the start (c) gives a similar solution at a fraction of the cost. Further reducing the localized grid to 36 nodes (d) still preserves the desired motion while greatly reducing the optimization cost. Note that the solution when forces are represented on particles (a), instead of a floating grid, results in inconsistencies between the tip and neighboring particles.
• Figure 3: Image-based keyframe control. Starting from an uncontrolled simulation (top left), users specify image-based keyframes as target shapes (top right). The control forces are optimized within a localized spacetime region, producing the animation from initial shapes to desired target shapes. By compositing the optimized control forces into the global simulation (bottom), the fluid naturally forms the specified shapes at the correct time steps while maintaining realistic dynamics.
• Figure 4: Evaluation of 2D localized splash editing under varying control parameters. Top row: The original simulation (left, blue) and a user-specified splash translation at a time in the middle of the simulation (right, red). (a) Comparison of different spatial control window sizes. Larger windows match the target better, while smaller windows (e.g., 5×5) struggle to achieve the desired effect. We can observe that once the spatial control grid is big enough to cover the region of interest, the control accuracy tends to converge to a satisfactory level. Objective function values are shown in the plots on the right. (b) Comparison of different spatial control grid spacings. Extremely fine grids will introduce high-frequency artifacts, while overly coarse grids lead to inconsistency among neighboring particles. The plot on the right illustrates the sweet spot that balances the trade-off. (c) Comparison of various temporal window sizes. Very short windows may cause impulsive motion or optimization failure, while overly long windows reduce control efficiency and make the optimization problem challenging. Searching with CMA-ES or approximating with spatial window size is crucial for effective control.
• Figure 5: Shape transformation from circles to letters F/L/U/I/D. Given five user-specified keyframes, we transform initial circles into target letters over 40 time steps. The prior Eulerian method pan-2017 (a) suffers from high-frequency artifacts, while tang-2021 (b) improves performance using reduced force representations. Recent work chen-2024 (c) employs the eigenfluid pipeline with the adjoint method to achieve smoother and faster smoke control, but remains grid-based and cannot handle free-surface flow. In contrast, our control pipeline (d) can be applied to both smoke and free-surface flows, achieving visually comparable results while maintaining efficiency on par with prior work chen-2024.