Rotate Your Character: Revisiting Video Diffusion Models for High-Quality 3D Character Generation

TL;DR

RCM tackles the challenge of turning single or multi-view images into high-quality 3D character representations by leveraging a video diffusion backbone augmented with a geometry-aware Camera Encoder and a progressive three-stage training strategy. It enables pose canonicalization to a canonical pose, high-resolution 1024×1024 video generation, controllable viewpoints, and multi-view conditioning with up to four inputs. The authors introduce two benchmarks, RCM-Wild and RCM-Hard, and show that RCM outperforms state-of-the-art diffusion approaches in both novel view synthesis and 3D character generation through comprehensive qualitative, quantitative, and ablation studies. The work advances practical, democratized 3D content creation and provides open resources for benchmarking and future research.

Abstract

Generating high-quality 3D characters from single images remains a significant challenge in digital content creation, particularly due to complex body poses and self-occlusion. In this paper, we present RCM (Rotate your Character Model), an advanced image-to-video diffusion framework tailored for high-quality novel view synthesis (NVS) and 3D character generation. Compared to existing diffusion-based approaches, RCM offers several key advantages: (1) transferring characters with any complex poses into a canonical pose, enabling consistent novel view synthesis across the entire viewing orbit, (2) high-resolution orbital video generation at 1024x1024 resolution, (3) controllable observation positions given different initial camera poses, and (4) multi-view conditioning supporting up to 4 input images, accommodating diverse user scenarios. Extensive experiments demonstrate that RCM outperforms state-of-the-art methods in both novel view synthesis and 3D generation quality.

Figures (7)

• Figure 1: Model Architecture and Training Strategies of the Proposed RCM. The proposed RCM is built upon video diffusion models and incorporates a Camera Encoder to achieve precise viewpoint control. To optimize learning, we adopt a progressive three‑stage training strategy, where each stage focuses respectively on pose canonicalization, viewpoint initialization, and character rotation.
• Figure 2: Training data filtering pipeline. We curated our dataset from a general pool, separating character and item data. Characters underwent quality-based filtering, retaining only high-quality geometry and textures. These were further categorized by style (realistic, cartoon/anime, and others) and augmented with randomized camera angles for viewpoint diversity.
• Figure 3: Qualitative comparisons with previous methods, including CharacterGen peng2024charactergen, Hi3D yang2024hi3d, AR-1-to-3 zhang2025ar, Wan 2.1 wan2025wan, SyncDreamer liu2023syncdreamer, SV3D voleti2024sv3d, Epidiff huang2024epidiff and Wan 2.2 wan2025wan. Our method effectively transformed characters with complex poses into standard A/T poses while preserving a complete orbital visualization of each character.
• Figure 4: Effects of different camera pose conditions. Users can specify arbitrary viewing angles—including top‑down, bottom‑up, and eye-level perspectives—with our method reliably generating geometrically accurate results that retain character appearance.
• Figure 5: Effects of multi-view images. Single-image input produced plausible completion of occluded regions (i.e., the first row); additional viewpoint images enabled geometric refinement through cross-view consensus (i.e., the second and the third rows). Note that in the first row, we prompted RCM to generate a feather pattern on the character's back.