MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphing

TL;DR

MorphAny3D tackles the problem of producing semantically coherent and temporally smooth 3D morphs without retraining. It leverages Trellis's Structured Latent (SLAT) as a robust prior and introduces Morphing Cross-Attention (MCA) and Temporal-Fused Self-Attention (TFSA) to fuse source/target information and past frames within the attention mechanism, coupled with an orientation correction strategy. The approach achieves state-of-the-art results across FID, PDV, and related metrics, and enables applications such as disentangled morphing and 3D style transfer while generalizing to other SLAT-based models. This framework advances practical 3D morphing by delivering high-quality, cross-category transitions efficiently and without additional training, with broad implications for 3D editing and stylization.

Abstract

3D morphing remains challenging due to the difficulty of generating semantically consistent and temporally smooth deformations, especially across categories. We present MorphAny3D, a training-free framework that leverages Structured Latent (SLAT) representations for high-quality 3D morphing. Our key insight is that intelligently blending source and target SLAT features within the attention mechanisms of 3D generators naturally produces plausible morphing sequences. To this end, we introduce Morphing Cross-Attention (MCA), which fuses source and target information for structural coherence, and Temporal-Fused Self-Attention (TFSA), which enhances temporal consistency by incorporating features from preceding frames. An orientation correction strategy further mitigates the pose ambiguity within the morphing steps. Extensive experiments show that our method generates state-of-the-art morphing sequences, even for challenging cross-category cases. MorphAny3D further supports advanced applications such as decoupled morphing and 3D style transfer, and can be generalized to other SLAT-based generative models. Project page: https://xiaokunsun.github.io/MorphAny3D.github.io/.

Figures (10)

• Figure 1: Comparison of different 3D morphing strategies. (a) Matching-Based 3D Morphing; (b) 2D Morphing + 3D Generation; (c) Direct Interpolation; (d) MorphAny3D. Our method leverages the powerful SLAT to achieve semantically plausible and temporally smooth 3D morphing without any training. $\alpha \in [0, 1]$ is the deformation weight controlling the morphing progress.
• Figure 2: (a) Overview of our method. MorphAny3D generates a smooth and high-quality morphing sequence between diverse object categories by leveraging the SLAT representation without any training. (b) Morphing Cross-Attention (MCA) fuses information from the source and target objects in the cross-attention layers to ensure the structural coherence and aesthetics of the deformation. (c) Temporal-Fused Self-Attention (TFSA) enhances temporal smoothness by incorporating SLAT features from the previous morphing frame into the self-attention mechanism, enabling smooth transitions over time. (d) An orientation correction strategy inspired by statistical orientation distribution patterns in Trellis-generated assets is proposed to resolve abrupt orientation shifts.
• Figure 3: Analysis of SLAT fusion patterns in attention for 3D morphing. (a) FID (plausibility) and PPL (smoothness) comparison. (b, c, d) Qualitative results of different fusion strategies (same case as \ref{['fig:comp']}).
• Figure 4: Attention maps visualization for different attention mechanisms. Red stars denote SLAT head features; pink stars mark their corresponding input regions. Orange boxes highlight KV-Fused CA's incorrect attention focus. MCA preserves correct, semantically consistent attention and avoids KV-Fused CA's artifacts shown in \ref{['fig:comp_fusion']}-(b).
• Figure 5: (a) Example of abrupt orientation change during morphing. (b) Distribution of $\alpha$ at orientation jumps, peaking near intermediate stages. (c) Adjacent-frame orientation changes $\Delta E$ at orientation jumps, dominated by 90°, 180°, and 270° yaw shifts. (d) Orientation distribution of Trellis-generated objects, showing non-canonical poses clustered at the same yaw angles.