MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices

TL;DR

This work tackles the challenge of real-time, high-resolution image-to-video generation on mobile devices by introducing MobileI2V, a 270M-parameter diffusion model that fuses linear and softmax attention in a hybrid DiT architecture and leverages a high compression ratio VAE to reduce token counts. A composite time-step distillation pipeline (regression, adversarial, and distribution-matching losses) compresses inference steps from roughly 20 to 1–2, enabling over 10× speedups on mobile hardware. Additional mobile-specific optimizations (4D channels-first layout, head tiling, and reduced data movement) yield roughly 2× speedups for attention computations, culminating in fast 720p I2V with frame times under 100 ms in certain settings. The system achieves 17 frames of 720p video in about 2 seconds on an iPhone 16 Pro and demonstrates competitive quality against cloud-based baselines, with practical on-device deployment demonstrated on Apple hardware. Limitations include potential blur in complex regions due to aggressive VAE compression and data limitations for broader scene diversity, motivating future work on higher-quality high-CR VAEs and more powerful linear attention variants.

Abstract

Recently, video generation has witnessed rapid advancements, drawing increasing attention to image-to-video (I2V) synthesis on mobile devices. However, the substantial computational complexity and slow generation speed of diffusion models pose significant challenges for real-time, high-resolution video generation on resource-constrained mobile devices. In this work, we propose MobileI2V, a 270M lightweight diffusion model for real-time image-to-video generation on mobile devices. The core lies in: (1) We analyzed the performance of linear attention modules and softmax attention modules on mobile devices, and proposed a linear hybrid architecture denoiser that balances generation efficiency and quality. (2) We design a time-step distillation strategy that compresses the I2V sampling steps from more than 20 to only two without significant quality loss, resulting in a 10-fold increase in generation speed. (3) We apply mobile-specific attention optimizations that yield a 2-fold speed-up for attention operations during on-device inference. MobileI2V enables, for the first time, fast 720p image-to-video generation on mobile devices, with quality comparable to existing models. Under one-step conditions, the generation speed of each frame of 720p video is less than 100 ms. Our code is available at: https://github.com/hustvl/MobileI2V.

Figures (16)

• Figure 1: Compared with SVD-XT ($1.5$B), our $5.55\times$ smaller MobileI2V ($0.27$B) achieves similar generation quality, using only $2.24$ s on mobile and running $199\times$ faster on an A100 GPU.
• Figure 2: Softmax Attention vs Linear Attention speed in mobile devices (Left). The I2V generation speed of the proposed model (Middle). Comparison results of existing I2V models (Right).
• Figure 3: Our proposed MobileI2V framework employs a hybrid attention framework to support fast inference, utilizes time-step distillation to reduce the number of inference steps, and finally optimizes the model on mobile devices to further accelerate inference.
• Figure 4: The proposed i2v time-step distillation framework. It includes three parts: regression loss, adversarial loss, and distribution loss.
• Figure 5: On-device latency on iPhone 16 Pro versus sequence length for linear attention (left) and softmax attention (right).