Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leum-VL Technical Report

Yuxuan He, Chaiming Huang, Yifan Wu, Hongjun Wang, Chenkui Shen, Jifan Zhang, Long Li

Abstract

A short video succeeds not simply because of what it shows, but because of how it schedules attention -- yet current multimodal models lack the structural grammar to parse or produce this organization. Existing models can describe scenes, answer event-centric questions, and read on-screen text, but they are far less reliable at identifying timeline-grounded units such as hooks, cut rationales, shot-induced tension, and platform-facing packaging cues. We propose SV6D (Structured Video in Six Dimensions), inspired by professional storyboard practice in film and television production, a representation framework that decomposes internet-native video into six complementary structural dimensions -- subject, aesthetics, camera language, editing, narrative, and dissemination -- with each label tied to physically observable evidence on the timeline. We formalize a unified optimization objective over SV6D that combines Hungarian-matched temporal alignment, dimension-wise semantic label distance, and quality regularization. Building on this framework, we present Leum-VL-8B, an 8B video-language model that realizes the SV6D objective through an expert-driven post-training pipeline, further refined through verifiable reinforcement learning on perception-oriented tasks. Leum-VL-8B achieves 70.8 on VideoMME (w/o subtitles), 70.0 on MVBench, and 61.6 on MotionBench, while remaining competitive on general multimodal evaluations such as MMBench-EN. We also construct FeedBench, a benchmark for structure-sensitive short-video understanding. Our results indicate that the missing layer in video AI is not pixel generation but structural representation: grounded on the timeline, linked to visible evidence, and directly consumable by downstream workflows such as editing, retrieval, recommendation, and generation control, including text-heavy internet video formats with overlays and image-text layouts.

Leum-VL Technical Report

Abstract

A short video succeeds not simply because of what it shows, but because of how it schedules attention -- yet current multimodal models lack the structural grammar to parse or produce this organization. Existing models can describe scenes, answer event-centric questions, and read on-screen text, but they are far less reliable at identifying timeline-grounded units such as hooks, cut rationales, shot-induced tension, and platform-facing packaging cues. We propose SV6D (Structured Video in Six Dimensions), inspired by professional storyboard practice in film and television production, a representation framework that decomposes internet-native video into six complementary structural dimensions -- subject, aesthetics, camera language, editing, narrative, and dissemination -- with each label tied to physically observable evidence on the timeline. We formalize a unified optimization objective over SV6D that combines Hungarian-matched temporal alignment, dimension-wise semantic label distance, and quality regularization. Building on this framework, we present Leum-VL-8B, an 8B video-language model that realizes the SV6D objective through an expert-driven post-training pipeline, further refined through verifiable reinforcement learning on perception-oriented tasks. Leum-VL-8B achieves 70.8 on VideoMME (w/o subtitles), 70.0 on MVBench, and 61.6 on MotionBench, while remaining competitive on general multimodal evaluations such as MMBench-EN. We also construct FeedBench, a benchmark for structure-sensitive short-video understanding. Our results indicate that the missing layer in video AI is not pixel generation but structural representation: grounded on the timeline, linked to visible evidence, and directly consumable by downstream workflows such as editing, retrieval, recommendation, and generation control, including text-heavy internet video formats with overlays and image-text layouts.
Paper Structure (42 sections, 17 equations, 3 figures, 6 tables, 1 algorithm)

This paper contains 42 sections, 17 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Structured Video in Six Dimensions (SV6D)
  3. Video as a Multimodal Time Series
  4. Temporal Primitives: Semantic Shots and Skeleton Segments
  5. Semantic Shots (Shot Tokens)
  6. Shot-Aligned Skeleton Segments (Discourse Structure)
  7. Six-Dimensional Schema
  8. SV6D Optimization Objective
  9. Temporal Alignment via Hungarian Matching
  10. Dimension-Wise Structural Matching
  11. Quality Regularization
  12. Summary
  13. Training Data and Synthesis
  14. SV6D Annotation
  15. Expert Annotation
  16. ...and 27 more sections

Figures (3)

  • Figure 1: Video understanding benchmark comparison of Leum-VL-8B against 8B-scale models.
  • Figure 2: Overview of the data synthesis pipeline. Starting from raw internet video, the pipeline proceeds through shot segmentation, LoRA-based structural annotation, scorer-based quality filtering, LLM-assisted structural extension, and dense report generation, producing the SV6D SFT training corpus.
  • Figure 3: GRPO training dynamics over 150 steps. Left: actor entropy decreases steadily, indicating the policy becomes more confident. Center: mean reward score rises from $\sim$0.66 to $\sim$0.80. Right: mean response length remains stable, showing no reward hacking through verbosity.