OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation

TL;DR

OpenS2V-Nexus provides a dedicated infrastructure for subject-to-video research by delivering OpenS2V-Eval, a fine-grained benchmark with 180 prompts across seven categories and three alignment metrics, plus OpenS2V-5M, a large-scale dataset featuring 5.1M regular samples and 0.35M Nexus Data generated via cross-video associations and GPT-Image-1. Together, these resources enable comprehensive evaluation of S2V models and support data-driven improvements to address generalization, copy-paste, and human-identity fidelity in open-domain and human-domain scenarios. The work also validates three new automatic metrics—NexusScore, NaturalScore, and GmeScore—against human judgments and demonstrates practical model selection guidance across a broad model landscape. By releasing both benchmarks and data openly, OpenS2V-Nexus aims to accelerate progress in subject-consistent video generation and facilitate downstream research and application development.

Abstract

Subject-to-Video (S2V) generation aims to create videos that faithfully incorporate reference content, providing enhanced flexibility in the production of videos. To establish the infrastructure for S2V generation, we propose OpenS2V-Nexus, consisting of (i) OpenS2V-Eval, a fine-grained benchmark, and (ii) OpenS2V-5M, a million-scale dataset. In contrast to existing S2V benchmarks inherited from VBench that focus on global and coarse-grained assessment of generated videos, OpenS2V-Eval focuses on the model's ability to generate subject-consistent videos with natural subject appearance and identity fidelity. For these purposes, OpenS2V-Eval introduces 180 prompts from seven major categories of S2V, which incorporate both real and synthetic test data. Furthermore, to accurately align human preferences with S2V benchmarks, we propose three automatic metrics, NexusScore, NaturalScore and GmeScore, to separately quantify subject consistency, naturalness, and text relevance in generated videos. Building on this, we conduct a comprehensive evaluation of 18 representative S2V models, highlighting their strengths and weaknesses across different content. Moreover, we create the first open-source large-scale S2V generation dataset OpenS2V-5M, which consists of five million high-quality 720P subject-text-video triples. Specifically, we ensure subject-information diversity in our dataset by (1) segmenting subjects and building pairing information via cross-video associations and (2) prompting GPT-Image-1 on raw frames to synthesize multi-view representations. Through OpenS2V-Nexus, we deliver a robust infrastructure to accelerate future S2V generation research.

Figures (24)

• Figure 1: Example of Seven Categories from OpenS2V-Eval. These categories fully encompass the subject-to-video tasks, allowing comprehensive evaluation. Videos are generated by Kling KeLing.
• Figure 2: The Pipeline of Constructing OpenS2V-Eval. (Left) Our benchmark includes not only real subject images but also synthetic images constructed through GPT-Image-1 gpt4, allowing for a more comprehensive evaluation. (Right) The metrics are tailored for subject-to-video generation, evaluating not only S2V characteristics (e.g., consistency) but also basic video elements (e.g., motion).
• Figure 3: Statistics in OpenS2V-Eval. The benchmark covers diverse categories and prompt words, with subject images displaying high aesthetics, thus enabling a thorough evaluation.
• Figure 4: The Pipeline of Constructing OpenS2V-5M. First, we filter low-quality videos based on scores such as aesthetics and motion, then utilize GroundingDino groundingdino and SAM2.1 sam2 to extract subject images and get Regular Data. Subsequently, we create Nexus Data through cross-video association and GPT-Image-1 gpt4 to address the three core issues encountered by S2V models.
• Figure 5: Comparison between Regular Data and Nexus Data. The latter is of higher quality.