ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusion

TL;DR

ActionMesh tackles the challenge of turning videos or prompts into production-ready animated 3D meshes by introducing a two-stage, feed-forward pipeline. It first applies a temporal 3D diffusion process to generate a sequence of time-varying 3D latents, then uses a temporal 3D autoencoder to deform a fixed reference mesh, ensuring topology consistency across frames. The approach leverages a VecSet-based 3D latent diffusion backbone with inflated attention and masked generation, enabling versatile inputs (video, text, images) and applications like text-to-4D and motion retargeting, with state-of-the-art geometric accuracy and temporal coherence and about a 10x speedup over prior methods. This work significantly accelerates production-quality animated 3D content while maintaining texture coherence and geometric fidelity, broadening practical 4D synthesis for games, AR/VR, and visualization.

Abstract

Generating animated 3D objects is at the heart of many applications, yet most advanced works are typically difficult to apply in practice because of their limited setup, their long runtime, or their limited quality. We introduce ActionMesh, a generative model that predicts production-ready 3D meshes "in action" in a feed-forward manner. Drawing inspiration from early video models, our key insight is to modify existing 3D diffusion models to include a temporal axis, resulting in a framework we dubbed "temporal 3D diffusion". Specifically, we first adapt the 3D diffusion stage to generate a sequence of synchronized latents representing time-varying and independent 3D shapes. Second, we design a temporal 3D autoencoder that translates a sequence of independent shapes into the corresponding deformations of a pre-defined reference shape, allowing us to build an animation. Combining these two components, ActionMesh generates animated 3D meshes from different inputs like a monocular video, a text description, or even a 3D mesh with a text prompt describing its animation. Besides, compared to previous approaches, our method is fast and produces results that are rig-free and topology consistent, hence enabling rapid iteration and seamless applications like texturing and retargeting. We evaluate our model on standard video-to-4D benchmarks (Consistent4D, Objaverse) and report state-of-the-art performances on both geometric accuracy and temporal consistency, demonstrating that our model can deliver animated 3D meshes with unprecedented speed and quality.

Figures (9)

• Figure 1: ActionMesh. Our model generates 3D meshes 'in action' from a wide range of inputs such as a text prompt, a video, an image + an animation text prompt, or a 3D mesh + an animation text prompt. Unlike previous approaches, our method is not only fast, but also rig-free and topology consistent. These properties are convenient in practice, e.g., they allow the seamless animation of complex 3D shapes like an octopus with maracas (top left) or the automatic transfer of the mesh texture throughout the animation (bottom right).
• Figure 2: Overview. Given an input video, our model generates an animated 3D mesh in two stages. (Stage I) After computing a reference mesh latent $\mathbf{z}_1^{*}$ with an off-the-shelf image-to-3D model, we use our temporal 3D diffusion model to produce from $\mathbf{z}_1^{*}$ and the video, a sequence of time-varying but independent 3D meshes. (Stage II) Our temporal 3D autoencoder takes theses shapes as input and predicts, for each shape in the sequence, a deformation field of the reference mesh vertices, thus yielding an animated 3D mesh with consistent topology.
• Figure 3: Image-to-3D results on video frames. Running an image-to-3D model on each frame produces meshes exhibiting inconsistent global orientation (left) or inconsistent geometric details (right), even when using identical Gaussian noise in the denoiser.
• Figure 4: Motion transfer results. Our model is able to accurately transfer the motion from a source video to target meshes, even if the objects are inconsistent.
• Figure 5: Qualitative comparison on Consistent4D jiang_consistent4d_2023. LIM sabathier2025lim and DM4D li2024dreammesh4d tend to produce coarse geometries that lack details. V2M4 chen2025v2m4 and SG4D yenphraphai2025shapegen4d recover sharper details but leave artifacts and partial drift. In contrast, our model preserves the highest geometric fidelity across frames with a strong temporal consistency. See our supplementary for additional examples.