PhysAnimator: Physics-Guided Generative Cartoon Animation

TL;DR

PhysAnimator tackles the labor-intensive process of hand-drawn anime animation by integrating image-space deformable-body dynamics with diffusion-based rendering. The pipeline segments and triangulates objects, simulates physically plausible motion under user-defined energy strokes and rigging points, warps sketches to produce dynamic sketches, and renders frames through a sketch-guided diffusion model, with an optional data-driven cartoon interpolation step. Key contributions include the deformable 2D mesh dynamics using Fixed Corotated energy, geometry registration via SAM, sketch-guided rendering with ControlNet, and complementary dynamics to enhance expressiveness. Results show improved visual quality, temporal consistency, and motion plausibility against baselines, indicating practical potential for producing controllable, high-fidelity anime-style animations from static illustrations.

Abstract

Creating hand-drawn animation sequences is labor-intensive and demands professional expertise. We introduce PhysAnimator, a novel approach for generating physically plausible meanwhile anime-stylized animation from static anime illustrations. Our method seamlessly integrates physics-based simulations with data-driven generative models to produce dynamic and visually compelling animations. To capture the fluidity and exaggeration characteristic of anime, we perform image-space deformable body simulations on extracted mesh geometries. We enhance artistic control by introducing customizable energy strokes and incorporating rigging point support, enabling the creation of tailored animation effects such as wind interactions. Finally, we extract and warp sketches from the simulation sequence, generating a texture-agnostic representation, and employ a sketch-guided video diffusion model to synthesize high-quality animation frames. The resulting animations exhibit temporal consistency and visual plausibility, demonstrating the effectiveness of our method in creating dynamic anime-style animations. See our project page for more demos: https://xpandora.github.io/PhysAnimator/

Figures (13)

• Figure 1: PhysAnimator is a novel framework that combines physics principles with video diffusion models to generate high-quality animations from static anime illustrations, allowing users to specify external forces or rigging points for custom effects.
• Figure 2: Method Overview. We begin by segmenting the object and creating a triangulated deformable mesh. Physics-based simulations are then used to generate dynamic optical flow fields, with users given the option to guide the motion through customizable energy strokes(shown as orange arrows) and rigging points(shown as red dots). The extracted sketch is warped using the computed optical flow and refined with a sketch-guided video diffusion model, producing a smooth, stylized animation sequence. Optionally, a cartoon interpolation model can further be applied to enhance the animation with expressive dynamics.
• Figure 3: Qualitative Comparison. We compare our results against Cinemo ma2024cinemo, Drag Anything wu2025draganything, DynamiCrafter xing2025dynamicrafter and Motion-I2V shi2024motion. Text prompts for Cinemo and DynamiCrafter are generated using ChatGPT-4V, while trajectories for Drag Anything and Motion-I2V are extracted from our animated results.
• Figure 4: Sketch-Guided Rendering. Applying warping and inpainting introduces artifacts due to segmentation inaccuracy. Soft-inpainting levin2023differential reduces these issues but can alter the content. Our sketch-guided rendering method produces high-quality results while preserving image details.
• Figure 5: Complementary Dynamics Enhancement. While physics-based animation maintains geometric consistency, it may lack the fluidity and exaggeration commonly seen in anime. We employ a data-driven interpolation module to enhance the motion dynamics, creating more natural-looking animations that better resemble real anime.