MVCustom: Multi-View Customized Diffusion via Geometric Latent Rendering and Completion

TL;DR

MVCustom introduces the problem of multi-view customization—simultaneous camera-pose control and subject customization—via a diffusion-based framework built on a pose-conditioned transformer and a video diffusion backbone with dense spatio-temporal attention. To cope with limited customization data, it employs two inference-time strategies: depth-aware feature rendering to enforce geometric consistency across new viewpoints, and consistent-aware latent completion to plausibly fill newly visible regions. Across CO3Dv2 and WebVid10M-derived setups, MVCustom outperforms baselines in both pose accuracy and cross-view coherence while preserving identity and aligning with textual prompts. The combination of tailored geometry-aware rendering and latent completion provides a practical path toward faithful, controllable multi-view generation in resource-constrained settings.

Abstract

Multi-view generation with camera pose control and prompt-based customization are both essential elements for achieving controllable generative models. However, existing multi-view generation models do not support customization with geometric consistency, whereas customization models lack explicit viewpoint control, making them challenging to unify. Motivated by these gaps, we introduce a novel task, multi-view customization, which aims to jointly achieve multi-view camera pose control and customization. Due to the scarcity of training data in customization, existing multi-view generation models, which inherently rely on large-scale datasets, struggle to generalize to diverse prompts. To address this, we propose MVCustom, a novel diffusion-based framework explicitly designed to achieve both multi-view consistency and customization fidelity. In the training stage, MVCustom learns the subject's identity and geometry using a feature-field representation, incorporating the text-to-video diffusion backbone enhanced with dense spatio-temporal attention, which leverages temporal coherence for multi-view consistency. In the inference stage, we introduce two novel techniques: depth-aware feature rendering explicitly enforces geometric consistency, and consistent-aware latent completion ensures accurate perspective alignment of the customized subject and surrounding backgrounds. Extensive experiments demonstrate that MVCustom is the only framework that simultaneously achieves faithful multi-view generation and customization.

Figures (7)

• Figure 1: Comparison between MVCustom and existing approaches extended to multi-view customization. The light blue box shows the reference multi-view images and corresponding camera poses of a customized object. The 'X' marks indicate regions inconsistent with either the reference object's appearance or across views, while 'O' marks indicate well-maintained consistency. Our approach clearly outperforms existing methods by achieving accurate viewpoint alignment and robust multi-view consistency for both the customized object and novel surroundings generated from diverse textual prompts.
• Figure 2: Overview. (a) The overall training pipeline, depicting how camera pose conditioning operates with two branches, the main and multi-view. (b) Visualization of our progressive attention mechanism. We gradually broaden the spatial attention field, enhancing geometric consistency. (c) The detailed illustration of the pose-conditioned transformer block. FeatureNeRF and a projection layer are trained to produce a feature map, obtained by concatenating the main-branch and multi-view feature map.
• Figure 3: (a) Anchor feature mesh $\mathcal{M}_a$, consists of a texture ${\bm{F}}_a$, vertices $\mathbf{P}_a$, and triangles $\mathcal{T}_a$, is constructed using the feature and depth maps, and camera pose of the anchor frame. The $\mathcal{M}_a$ is used to render the projected feature maps for the other camera poses. (b) Completion via latent perturbation for new visible areas.
• Figure 4: Qualitative results. The light blue boxes indicate the multi-view training dataset for the target concept, while the light pink boxes illustrate the inference phase, where results are conditioned on new text and target camera poses.
• Figure 5: Results of ablation studies. (a) Stepwise effect of applying depth-aware feature rendering (DFR) and consistent-aware latent completion under x-translation camera pose. (b) Impact of temporal attention on feature replacement. (i) Feature replacement vertically copies the feature map from frame 1 to frame 2. Our method successfully enforces spatial flow, whereas 1D temporal attention fails to capture the intended translation.