HiVid-Narrator: Hierarchical Video Narrative Generation with Scene-Primed ASR-anchored Compression

TL;DR

This work tackles the challenge of generating structured, fact-grounded narratives for dense e-commerce videos by introducing a dual-granularity annotation scheme (Temporal Chain-of-Thought and Chapter Summary) within the E-HVC dataset and a hierarchical narrative framework, HiVid-Narrator. A key contribution is the Scene-Primed ASR-anchored Compressor (SPA-Compressor), which compresses long multimodal inputs into a compact hierarchy of scene and event tokens guided by ASR cues, enabling efficient training and inference. Empirical results show state-of-the-art performance on dense video caption benchmarks and substantial token savings (up to $82.59\%$) without sacrificing narrative quality, validating the coarse-to-fine reasoning and scene-aware representation. The work advances practical e-commerce video understanding by delivering accurate, coherent chapters and event-level details with improved factual grounding and scalability for real-world deployment.

Abstract

Generating structured narrations for real-world e-commerce videos requires models to perceive fine-grained visual details and organize them into coherent, high-level stories--capabilities that existing approaches struggle to unify. We introduce the E-commerce Hierarchical Video Captioning (E-HVC) dataset with dual-granularity, temporally grounded annotations: a Temporal Chain-of-Thought that anchors event-level observations and Chapter Summary that compose them into concise, story-centric summaries. Rather than directly prompting chapters, we adopt a staged construction that first gathers reliable linguistic and visual evidence via curated ASR and frame-level descriptions, then refines coarse annotations into precise chapter boundaries and titles conditioned on the Temporal Chain-of-Thought, yielding fact-grounded, time-aligned narratives. We also observe that e-commerce videos are fast-paced and information-dense, with visual tokens dominating the input sequence. To enable efficient training while reducing input tokens, we propose the Scene-Primed ASR-anchored Compressor (SPA-Compressor), which compresses multimodal tokens into hierarchical scene and event representations guided by ASR semantic cues. Built upon these designs, our HiVid-Narrator framework achieves superior narrative quality with fewer input tokens compared to existing methods.

Figures (4)

• Figure 1: An annotation example from E-HVC. The model takes video frames and ASR transcripts as input, and generates hierarchical outputs: event-level Temporal Chain-of-Thought, followed by chapter-based Dense Video Captions with corresponding chapter titles.
• Figure 2: Multi-stage annotation pipeline for E-HVC-146K. (a) Multi-level ASR text quality enhancement; (b) Temporally aligned frame-level description generation; (c) Hierarchical reasoning from coarse to fine-grained annotations.
• Figure 3: (a) Architecture of HiVid-Narrator. The model processes four types of inputs: system prompt tokens, timestamp tokens, vision tokens, and ASR text tokens. These are compressed by the SPA-Compressor module before being fed into the LLM along with scene context tokens to generate event context tokens with timestamps. (b) Architecture of SPA-Compressor. The module consists of three stages: Fusion Block fuses vision and ASR tokens, SceneFusionAggregator extracts scene-level context via cross-attention with scene query tokens, and EventDetailExtractor generates event-level representations with timestamp tokens through transformer decoder blocks.
• Figure 4: E-HVC-Bench: Distribution of 1,852 benchmark videos across 13 categories.