AniClipart: Clipart Animation with Text-to-Video Priors

TL;DR

This work addresses the labor-intensive process of animating static clipart by leveraging pretrained text-to-video priors. It introduces AniClipart, which parameterizes keypoint motion with cubic Bézier curves and optimizes them under a video SDS loss plus a skeleton fidelity term, enabled by differentiable ARAP deformation and rendering to preserve structure and dynamics. The approach shows superior identity preservation, text-video alignment, and temporal coherence compared with 2D sketch animation and standard T2V methods, and supports layered/topology-changing animations. The method reduces manual animation effort while delivering high-quality, cartoon-like motion, with practical potential for SVG and vector clipart workflows; future work includes automated keypoint detection and layering, and 2.5D extensions for richer scene dynamics.

Abstract

Clipart, a pre-made art form, offers a convenient and efficient way of creating visual content. However, traditional workflows for animating static clipart are laborious and time-consuming, involving steps like rigging, keyframing, and inbetweening. Recent advancements in text-to-video generation hold great potential in resolving this challenge. Nevertheless, direct application of text-to-video models often struggles to preserve the visual identity of clipart or generate cartoon-style motion, resulting in subpar animation outcomes. In this paper, we introduce AniClipart, a computational system that converts static clipart into high-quality animations guided by text-to-video priors. To generate natural, smooth, and coherent motion, we first parameterize the motion trajectories of the keypoints defined over the initial clipart image by cubic Bézier curves. We then align these motion trajectories with a given text prompt by optimizing a video Score Distillation Sampling (SDS) loss and a skeleton fidelity loss. By incorporating differentiable As-Rigid-As-Possible (ARAP) shape deformation and differentiable rendering, AniClipart can be end-to-end optimized while maintaining deformation rigidity. Extensive experimental results show that the proposed AniClipart consistently outperforms the competing methods, in terms of text-video alignment, visual identity preservation, and temporal consistency. Additionally, we showcase the versatility of AniClipart by adapting it to generate layered animations, which allow for topological changes.

Figures (14)

• Figure 1: AniClipart creates high-quality clipart animations guided by text prompts with visual identity preservation and temporal motion consistency. The left panel displays different animations generated from the same clipart input, each guided by a different text prompt. The right panel presents animations across diverse clipart categories. Initial clipart images are marked with dashed-line boxes.
• Figure 2: A simplified animation production pipeline.
• Figure 3: System Diagram of AniClipart. Given an initial clipart image with $M$ keypoints, we initialize $M$ corresponding cubic Bézier motion trajectories, parameterized by $\{c^{(i)}\}_{i=0}^{M-1}$. For a sequence of $N$ frames, keypoints are updated at each frame by sampling along these trajectories. The displaced keypoints are responsible for driving the ARAP shape deformation algorithm, which warps the object, represented by a triangle mesh, into new poses. This gives rise to a clipart animation, which is (optionally rasterized and) passed to a T2V model to compute the video SDS loss. To ensure motion coherence across all keypoints, a skeleton fidelity loss is also applied, penalizing changes in bone lengths over time.
• Figure 4: Template-based and anatomically meaningful keypoint detection by UniPose yang2023unipose for articulated objects (e.g., humans), followed by skeletonization and triangulation.
• Figure 5: Template-free keypoint detection and skeleton construction. The first row shows a visual example of an invertebrate starfish, while the second row highlights the impact of different $\rho$ values.