Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MUG-V 10B: High-efficiency Training Pipeline for Large Video Generation Models

Yongshun Zhang, Zhongyi Fan, Yonghang Zhang, Zhangzikang Li, Weifeng Chen, Zhongwei Feng, Chaoyue Wang, Peng Hou, Anxiang Zeng

TL;DR

The work targets the bottlenecks in training large-scale video generation models, notably cross-modal alignment, long sequence handling, and spatiotemporal fidelity. It delivers an end-to-end framework (MUG-V 10B) built on scalable data processing, a high-compression Video VAE, a 10B Diffusion Transformer, a multi-stage pre-training curriculum, and post-training alignment, all implemented within Megatron-Core for efficient, near-linear multi-node scaling. Empirically, MUG-V 10B achieves competitive or superior performance to state-of-the-art baselines and shows strong results in e-commerce-specific video generation, supported by both automatic metrics (VBench) and human evaluations, while open-sourcing the full stack to accelerate community advancement. The combination of aggressive latent compression, cross-modal conditioning, curriculum-based training, and system-level optimizations yields substantial training efficiency gains and practical appeal for real-world video synthesis at scale, albeit with remaining gaps in fine-grained texture fidelity and long-duration coherence.

Abstract

In recent years, large-scale generative models for visual content (\textit{e.g.,} images, videos, and 3D objects/scenes) have made remarkable progress. However, training large-scale video generation models remains particularly challenging and resource-intensive due to cross-modal text-video alignment, the long sequences involved, and the complex spatiotemporal dependencies. To address these challenges, we present a training framework that optimizes four pillars: (i) data processing, (ii) model architecture, (iii) training strategy, and (iv) infrastructure for large-scale video generation models. These optimizations delivered significant efficiency gains and performance improvements across all stages of data preprocessing, video compression, parameter scaling, curriculum-based pretraining, and alignment-focused post-training. Our resulting model, MUG-V 10B, matches recent state-of-the-art video generators overall and, on e-commerce-oriented video generation tasks, surpasses leading open-source baselines in human evaluations. More importantly, we open-source the complete stack, including model weights, Megatron-Core-based large-scale training code, and inference pipelines for video generation and enhancement. To our knowledge, this is the first public release of large-scale video generation training code that exploits Megatron-Core to achieve high training efficiency and near-linear multi-node scaling, details are available in https://github.com/Shopee-MUG/MUG-V.

MUG-V 10B: High-efficiency Training Pipeline for Large Video Generation Models

TL;DR

The work targets the bottlenecks in training large-scale video generation models, notably cross-modal alignment, long sequence handling, and spatiotemporal fidelity. It delivers an end-to-end framework (MUG-V 10B) built on scalable data processing, a high-compression Video VAE, a 10B Diffusion Transformer, a multi-stage pre-training curriculum, and post-training alignment, all implemented within Megatron-Core for efficient, near-linear multi-node scaling. Empirically, MUG-V 10B achieves competitive or superior performance to state-of-the-art baselines and shows strong results in e-commerce-specific video generation, supported by both automatic metrics (VBench) and human evaluations, while open-sourcing the full stack to accelerate community advancement. The combination of aggressive latent compression, cross-modal conditioning, curriculum-based training, and system-level optimizations yields substantial training efficiency gains and practical appeal for real-world video synthesis at scale, albeit with remaining gaps in fine-grained texture fidelity and long-duration coherence.

Abstract

In recent years, large-scale generative models for visual content (\textit{e.g.,} images, videos, and 3D objects/scenes) have made remarkable progress. However, training large-scale video generation models remains particularly challenging and resource-intensive due to cross-modal text-video alignment, the long sequences involved, and the complex spatiotemporal dependencies. To address these challenges, we present a training framework that optimizes four pillars: (i) data processing, (ii) model architecture, (iii) training strategy, and (iv) infrastructure for large-scale video generation models. These optimizations delivered significant efficiency gains and performance improvements across all stages of data preprocessing, video compression, parameter scaling, curriculum-based pretraining, and alignment-focused post-training. Our resulting model, MUG-V 10B, matches recent state-of-the-art video generators overall and, on e-commerce-oriented video generation tasks, surpasses leading open-source baselines in human evaluations. More importantly, we open-source the complete stack, including model weights, Megatron-Core-based large-scale training code, and inference pipelines for video generation and enhancement. To our knowledge, this is the first public release of large-scale video generation training code that exploits Megatron-Core to achieve high training efficiency and near-linear multi-node scaling, details are available in https://github.com/Shopee-MUG/MUG-V.
Paper Structure (39 sections, 3 equations, 7 figures, 2 tables, 1 algorithm)

This paper contains 39 sections, 3 equations, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Data Processing
  3. Scalable Video Processing Pipeline
  4. Human-Labelled Post-training Data
  5. High-quality Clip Labeling
  6. Preference Optimisation Data Labeling
  7. Model Design and Architecture
  8. Video Variational Autoencoder (Video VAE)
  9. Video VAE Architecture
  10. Minimal Encoding Strategy
  11. MUG-V 10B Diffusion Transformer Model
  12. Model Training
  13. Video VAE Training
  14. Adaptive Reconstruction Weighting.
  15. MUG-V 10B Diffusion Transformer Training
  16. ...and 24 more sections

Figures (7)

  • Figure 1: The visualization of Video VAE reconstruction examples. For each example, we provide the input video frame (the whole frame and local details) and the local patch extracted from the reconstructed video clip (the right enlarge part).
  • Figure 2: The human evaluation comparisons on generated e-commerce video of their quality.
  • Figure 3: Visualization of text-to-video generation results produced by the MUG-V 10B model. (enlarge for more details.)
  • Figure 4: Visualization of image-to-video generation results produced by the MUG-V 10B model. In each example, the first frame corresponds to the conditioning image. (enlarge for more details.)
  • Figure 5: Visualization of e-commerce video generation results across different models. Since each model is optimized for distinct prompt styles, we employed their respective default prompts or prompt-rewriting tools for fair comparison. (enlarge for more details.)
  • ...and 2 more figures