Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Think

TL;DR

This work tackles the challenge of motion modeling in image-to-video generation by introducing an Extrapolating and Decoupling framework that merges models to separately learn motion controllability and motion degree. The approach comprises three stages: (1) injecting text into the temporal module to improve control, (2) a training-free extrapolation that amplifies motion dynamics, and (3) decoupled injection of distinct parameter sets at different denoising steps to balance motion and consistency. The method demonstrates substantial gains on VBench and UCF101 benchmarks, outperforming prior I2V methods and even large pretrained baselines, while maintaining video quality and appearance fidelity. By applying model merging and time-aware parameter decoupling to video diffusion, the work provides a practical path to richer, more controllable motion in I2V generation with minimal additional training and clear interpretability of the knowledge being transferred.

Abstract

Image-to-Video (I2V) generation aims to synthesize a video clip according to a given image and condition (e.g., text). The key challenge of this task lies in simultaneously generating natural motions while preserving the original appearance of the images. However, current I2V diffusion models (I2V-DMs) often produce videos with limited motion degrees or exhibit uncontrollable motion that conflicts with the textual condition. To address these limitations, we propose a novel Extrapolating and Decoupling framework, which introduces model merging techniques to the I2V domain for the first time. Specifically, our framework consists of three separate stages: (1) Starting with a base I2V-DM, we explicitly inject the textual condition into the temporal module using a lightweight, learnable adapter and fine-tune the integrated model to improve motion controllability. (2) We introduce a training-free extrapolation strategy to amplify the dynamic range of the motion, effectively reversing the fine-tuning process to enhance the motion degree significantly. (3) With the above two-stage models excelling in motion controllability and degree, we decouple the relevant parameters associated with each type of motion ability and inject them into the base I2V-DM. Since the I2V-DM handles different levels of motion controllability and dynamics at various denoising time steps, we adjust the motion-aware parameters accordingly over time. Extensive qualitative and quantitative experiments have been conducted to demonstrate the superiority of our framework over existing methods.

Figures (8)

• Figure 1: In the first example, ConsistI2V ren2024consisti2v yields a video with limited motion. In the second, DynamiCrafter xing2025dynamicrafter fails to follow the text condition and lacks controllability. Our method, however, achieves improved motion degree and better controllability.
• Figure 2: The overview of our framework, consisting of three stages. (a) We improve the motion controllability by injecting the textual conditions into the temporal attention module in I2V-DM model and then fine-tune the integrated model. (b) The I2V-DM model suffers from a decrease in motion degree after fine-tuning. To mitigate it, we propose a training-free extrapolation to boost the motion degree by reversing the fine-tuning progress. (c) We decouple the relevant parameters contributing to motion controllability and motion degree from the last two-stage models. Moreover, we selectively inject these parameters into the I2V model along with the de-noising process.
• Figure 3: Human Evaluation of our method Compared to SVD, VideoCrafter (VC), DynamiCrafter (DC), and its variant of Native Fine-Tuned (DC-FT) and CIL-Based Methods.
• Figure 4: In the visual comparisons, we generated 16-frame videos using DynamiCrafter, CIL, Consistent-I2V, and our model. We selected 3 to 4 frames at regular intervals for display at the top. The upper section visualizes the overall aesthetic quality, while the lower section uses Optical Flow to illustrate the dynamic qualities of the generated videos for each method.
• Figure 5: The impact of different extrapolate strengths and different sampling strategies.