Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SeriesBench: A Benchmark for Narrative-Driven Drama Series Understanding

Chenkai Zhang, Yiming Lei, Zeming Liu, Haitao Leng, Shaoguo Liu, Tingting Gao, Qingjie Liu, Yunhong Wang

TL;DR

SeriesBench addresses the gap in narrative understanding for drama series by providing a long-span, multi-modal benchmark built from 105 series (1,072 videos) with 28 tasks across five dimensions. The authors introduce PC-DCoT, which constructs and integrates Plot Event Chains and Character Temporal Chains to guide reasoning over extended narratives, significantly boosting performance for Video-MLLMs and revealing persistent gaps to human-level understanding. Across open-source and commercial models, results show that current systems struggle with narrative structure yet benefit from the dual-chain approach, with gains exceeding 10 percentage points in many settings. The dataset and PC-DCoT framework offer a resource and methodology to advance long-form narrative understanding in practical applications like series recommendation, summarization, and interactive media.

Abstract

With the rapid development of Multi-modal Large Language Models (MLLMs), an increasing number of benchmarks have been established to evaluate the video understanding capabilities of these models. However, these benchmarks focus on standalone videos and mainly assess "visual elements" like human actions and object states. In reality, contemporary videos often encompass complex and continuous narratives, typically presented as a series. To address this challenge, we propose SeriesBench, a benchmark consisting of 105 carefully curated narrative-driven series, covering 28 specialized tasks that require deep narrative understanding. Specifically, we first select a diverse set of drama series spanning various genres. Then, we introduce a novel long-span narrative annotation method, combined with a full-information transformation approach to convert manual annotations into diverse task formats. To further enhance model capacity for detailed analysis of plot structures and character relationships within series, we propose a novel narrative reasoning framework, PC-DCoT. Extensive results on SeriesBench indicate that existing MLLMs still face significant challenges in understanding narrative-driven series, while PC-DCoT enables these MLLMs to achieve performance improvements. Overall, our SeriesBench and PC-DCoT highlight the critical necessity of advancing model capabilities to understand narrative-driven series, guiding the future development of MLLMs. SeriesBench is publicly available at https://github.com/zackhxn/SeriesBench-CVPR2025.

SeriesBench: A Benchmark for Narrative-Driven Drama Series Understanding

TL;DR

SeriesBench addresses the gap in narrative understanding for drama series by providing a long-span, multi-modal benchmark built from 105 series (1,072 videos) with 28 tasks across five dimensions. The authors introduce PC-DCoT, which constructs and integrates Plot Event Chains and Character Temporal Chains to guide reasoning over extended narratives, significantly boosting performance for Video-MLLMs and revealing persistent gaps to human-level understanding. Across open-source and commercial models, results show that current systems struggle with narrative structure yet benefit from the dual-chain approach, with gains exceeding 10 percentage points in many settings. The dataset and PC-DCoT framework offer a resource and methodology to advance long-form narrative understanding in practical applications like series recommendation, summarization, and interactive media.

Abstract

With the rapid development of Multi-modal Large Language Models (MLLMs), an increasing number of benchmarks have been established to evaluate the video understanding capabilities of these models. However, these benchmarks focus on standalone videos and mainly assess "visual elements" like human actions and object states. In reality, contemporary videos often encompass complex and continuous narratives, typically presented as a series. To address this challenge, we propose SeriesBench, a benchmark consisting of 105 carefully curated narrative-driven series, covering 28 specialized tasks that require deep narrative understanding. Specifically, we first select a diverse set of drama series spanning various genres. Then, we introduce a novel long-span narrative annotation method, combined with a full-information transformation approach to convert manual annotations into diverse task formats. To further enhance model capacity for detailed analysis of plot structures and character relationships within series, we propose a novel narrative reasoning framework, PC-DCoT. Extensive results on SeriesBench indicate that existing MLLMs still face significant challenges in understanding narrative-driven series, while PC-DCoT enables these MLLMs to achieve performance improvements. Overall, our SeriesBench and PC-DCoT highlight the critical necessity of advancing model capabilities to understand narrative-driven series, guiding the future development of MLLMs. SeriesBench is publicly available at https://github.com/zackhxn/SeriesBench-CVPR2025.
Paper Structure (37 sections, 14 equations, 15 figures, 33 tables)

This paper contains 37 sections, 14 equations, 15 figures, 33 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. SeriesBench
  4. Task Dimension Definitions
  5. Task Annotation Design
  6. Long-span Narrative Annotation
  7. Full-Information Transformation
  8. Comparison with Previous Benchmarks
  9. Plot & Character Dual Chain of Thought
  10. Event and Character Extraction.
  11. Plot and Character Chains Construction.
  12. Dual-Chain Synthesis and Reasoning.
  13. Experiments
  14. Settings
  15. Main Results on SeriesBench
  16. ...and 22 more sections

Figures (15)

  • Figure 1: An example from SeriesBench. The task involves multiple events and characters spanning a long time period in a video.
  • Figure 2: Illustration of PC-DCoT. The process begins by extracting relevant events and characters from the input question and the original video. Subsequently, a Plot Event Chain and a Character Temporal Chain are constructed independently. Finally, these chains are merged, enabling a reasoning process that generates an answer to the posed question based on the integrated dual-chain framework.
  • Figure 3: Task Dimension in SeriesBench. Detailed sample count for each task in SeriesBench.
  • Figure 4: Video Categories in SeriesBench. The videos in SeriesBench are categorized into two main types: thematic videos and series videos. These encompass 11 of the most popular video themes, including Urban Life, Romance, Fantasy, Counterattack, Family, Ancient Style, Campus Life, Anime, Funny Daily, Short Drama, and Food. Each series is accompanied by a number indicating the total count of videos within that series.
  • Figure 5: Illustration of the web-based annotation interface. Displaying parsed video information for reference, a hierarchical video index for navigation, and an annotation area for detailed content analysis and categorization.
  • ...and 10 more figures