Let Your Image Move with Your Motion! -- Implicit Multi-Object Multi-Motion Transfer

TL;DR

FlexiMMT is presented, the first implicit image-to-video (I2V) motion transfer framework that explicitly enables multi-object, multi-motion transfer and introduces a Motion Decoupled Mask Attention Mechanism, which derives object-specific masks directly from diffusion attention and progressively propagates them across frames efficiently.

Abstract

Motion transfer has emerged as a promising direction for controllable video generation, yet existing methods largely focus on single-object scenarios and struggle when multiple objects require distinct motion patterns. In this work, we present FlexiMMT, the first implicit image-to-video (I2V) motion transfer framework that explicitly enables multi-object, multi-motion transfer. Given a static multi-object image and multiple reference videos, FlexiMMT independently extracts motion representations and accurately assigns them to different objects, supporting flexible recombination and arbitrary motion-to-object mappings. To address the core challenge of cross-object motion entanglement, we introduce a Motion Decoupled Mask Attention Mechanism that uses object-specific masks to constrain attention, ensuring that motion and text tokens only influence their designated regions. We further propose a Differentiated Mask Propagation Mechanism that derives object-specific masks directly from diffusion attention and progressively propagates them across frames efficiently. Extensive experiments demonstrate that FlexiMMT achieves precise, compositional, and state-of-the-art performance in I2V-based multi-object multi-motion transfer.

Figures (11)

• Figure 1: Illustration of FlexiMMT. Given multiple reference videos, our FlexiMMT independently extracts each motion and applies them to images with any number of objects, enabling precise and compositional multi-object multi-motion transfer. For clarity, we show only simplified motion labels here; full text prompts and vivid videos are provided in the Supplementary Material.
• Figure 2: Overview of FlexiMMT. (a) Training: Given one reference video, insert trainable motion tokens into text and video tokens. Get the object mask through a simple $QK$ multiplication method, then mask out M2M, M2V and T2M parts in attention map. (b) Inference: Given a multi-object conditional image, we first segment each object's mask with semantic segmentation model groundedsam. Concatenate pre-trained motion tokens into text and video tokens for inference. Extract each object's latent-space mask in subsequent frames via Dynamic Regressive Mask Propagation Mechanism (Dynamic RMPM), and apply it to Motion parts and Text parts in attention map.
• Figure 3: Overview of the simple method for training and inference. Single object can be extracted through a simple $QK$ multiplication method. However, different objects cannot be extracted through this simple method due to feature entanglement.
• Figure 4: Illustration of the propagate feature and mask changes during denoising steps. We found that the transfer of motion can be completed in the early stage of denoising steps.
• Figure 5: Qualitative comparison. We compared the effects of different methods on transferring multi-object motions. The first row is the referenced motion, and the next six rows are the results of different methods.