MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation

TL;DR

This work introduces MJ-Bench-Video, a large-scale, fine-grained video preference benchmark across five aspects with 28 criteria to robustly evaluate video reward models. It also proposes MJ-Video, a Mixture-of-Experts reward model built on VideoLLM that decomposes video assessment into Aspect MoE and Criteria MoE, trained in three stages to deliver precise, adaptable judgments. Experimental results demonstrate MJ-Video's superiority over existing judges on multiple datasets and its effectiveness in improving text-to-video alignment when used in reward-based tuning. The combination of fine-grained annotations and MoE-based routing enables more reliable and nuanced video preference judgments, advancing the development of alignment-aware video generation systems.

Abstract

Recent advancements in video generation have significantly improved the ability to synthesize videos from text instructions. However, existing models still struggle with key challenges such as instruction misalignment, content hallucination, safety concerns, and bias. Addressing these limitations, we introduce MJ-BENCH-VIDEO, a large-scale video preference benchmark designed to evaluate video generation across five critical aspects: Alignment, Safety, Fineness, Coherence & Consistency, and Bias & Fairness. This benchmark incorporates 28 fine-grained criteria to provide a comprehensive evaluation of video preference. Building upon this dataset, we propose MJ-VIDEO, a Mixture-of-Experts (MoE)-based video reward model designed to deliver fine-grained reward. MJ-VIDEO can dynamically select relevant experts to accurately judge the preference based on the input text-video pair. This architecture enables more precise and adaptable preference judgments. Through extensive benchmarking on MJ-BENCH-VIDEO, we analyze the limitations of existing video reward models and demonstrate the superior performance of MJ-VIDEO in video preference assessment, achieving 17.58% and 15.87% improvements in overall and fine-grained preference judgments, respectively. Additionally, introducing MJ-VIDEO for preference tuning in video generation enhances the alignment performance. All our code, data, and models are available at https://aiming-lab.github.io/MJ-VIDEO.github.io/.

Figures (9)

• Figure 1: MJ-Bench-Video is a comprehensive and fine-grained large-scale video preference dataset, which includes five aspects: Alignment, Safety, Fineness, Coherence and Consistency (C&C), and Bias and Fairness (B&F). Each aspect contains multiple detailed criteria to facilitate a thorough preference evaluation from different perspectives.
• Figure 2: MJ-Bench-Video curation process consists of three stages: data collection, data filtering, and data annotation.
• Figure 3: The structure of MJ-Video which builds upon a VideoLLM and consists of two stacked MoE layers. The first MoE layer is for aspect routing and the second one is for scoring each fine-grained criteria. An overall score is also offered by weighting those scores.
• Figure 4: (a): Compare MJ-Video with "w/o Criteria MoE", where average results of Acc, F1, and strict metrics are evaluated over five aspects; (b) Compare MJ-Video with "w/o Aspect MoE" on MJ-Bench-Video Safesora-test and GenAI-Bench.
• Figure 5: Two cases of video preference analysis.